For questions like this I recommend the Nielsen Norman Group, as it's a great source of research-based usability guidelines.
They also covered both back-to-top buttons and chats:
Back-to-Top Button Design Guidelines
The User Experience of Customer-Service Chat: 20 Guidelines
It appears that they should indeed both be placed on the right side of the page.
Test with your prospective users.
As you suggest, both sides have pros and cons but the best way to settle the matter is to get a bunch of users to test a prototype.
Taking control of the browser's back button can be very productive in situations where the user's view changes to the point where they feel like they are on a new page.
I would suggest ...
Provide your user with determinable action controls, that is, appropriate controls to remove filters that have been added and a navigation control to allow the user to go to the pages and locations they need to use your application.
Leave the browser back-button alone:
Breaking or Slowing Down the Back Button
The Back button is the ...