This is very nonstandard on the web - either the left mouse button drag-scrolls or it drag-selects, depending on the type of element, and the right mouse button pops up a menu. Example: in google maps the left button scrolls the map element (div) and in all the other places on the page has the more conventional behavior of selecting text or dragging links. (This was very disconcerting at first because it was nonstandard, but google maps has become ubiquitous enough so that it is not longer unconventional.)
There are some ways to have both drag-select and drag-scroll in the same element: the right-button drag you suggest is probably the least unconventional, and it would be the approach I would use.
But it still is unconventional, and it does have the downside of displacing the popup menu, and that may disturb some users who happen to use the popup menu. There are various key-mouse-button combinations that could also be used: control-left-button, shift-right-button, etc. but many of these combinations already have an established behavior you'd be overriding.
In any case whatever you do to have both drag-select and drag-scroll in the same element will be unconventional, so you should do whatever you can to mitigate this, including explicit statements ("Right mouse button will drag"), pointer changes for feedback, etc.