Yes, you do.
Something is upsetting a significant portion of your users. The experience they are having is, to put it politely, sub-optimal. Therefore you have a User Experience problem.
I suspect however you were actually asking if you had a UI problem.
Without more information it's very hard to know what the problem actually is. But here are some likely candidates.
1) Users can't achieve what they want or expect to.
You have a UI problem. Usability test your product.
2) It doesn't solve your users problems.
Do some user research find out what they really want.
3) Users are buying / downloading by mistake.
You have a communication problem. Your product description may sucks or you're making claims that are not true.
You need more data. Are there any written reviews that can give you clues? Have you done any usability testing? Have you surveyed your users?
Remember the best way to find out what users really think is to ask them. Concentrate on the unhappy people, they will tell you much much more than the fanboys.