We have a mobile app for different platforms. And now we want to improve the UX and UI. Due to the security policy, we cannot install analytic tools. I'm thinking of options we have to measure our design improvements impact: One-on-one A/B testing with 20 users per design to measure speed, number of errors, satisfaction rate etc. That's about 40 user tests per design update, which seems to be not realistically for our company. Any more variants or thoughts? Once in a few months we have 1 negative feedback about the design like "I don't like it" and some more feedback like "I like it".
we use SUS questionnaire for our usability test in a company. It contains 10 questions which contains negative as well as positive feedback. The more information might be helpful here: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html.
It's too bad you can't place Google Analytics, or similar traffic analysis tool, on your mobile app. Usage is always your best indicator of utility.
Many of the video testing platforms now include space for opinion polls or SUS scores. These can provide a quantitative context for what is a very qualitative way of assessing usability.
The video recordings, using "think-aloud" narration, are an excellent way to identify errors in mobile UI. If you haven't tried them out yet, demo UserTesting, Userlytics or UserZoom for video testing your mobile app.