When you say that allowing the users to manually enter values "doesn't seem like good UX", it is important to keep in mind that this is going to be context dependent, and some of this information is missing in the question. The only thing mentioned is that the accuracy is not crucial but the more accurate the better (is that from the user's point of view?).
If we assume this to be true from the user's perspective, then manual entry does allow for the most accurate input. However, we need to balance this with what the purpose of the chart is, and whether there is an interaction between the input and the chart that helps to improve the outcome of the task they are trying to perform.
For example, if there is feedback from the chart as a result of the input (e.g. you zoom into the specific region) then it is possible that direct input on the chart (e.g. drag around a region) can be more intuitive. However, if this is on a mobile device then action of the input itself could be difficult, and the navigation afterwards also awkward.
So the point is that you need to define the task and the expectations of the user before you can determine whether the experience is going to be optimal. Something that helps you determine this would be looking at the frequency of the different types of input and work out the worst case scenarios. If you can provide a good experience most of the time, and the impact of the bad experience doesn't lead to severe repercussions then that's not a bad starting point.