Microsoft's Desirability Testing framework might be a helpful method. You would show your research participants a set of positive and negative words and ask them to pick the five words that most closely describe your app.
You would then cluster the results and rank them by frequency.
From Nielsen-Norman Group:
To analyze your participants’ responses, determine the percentage of participants who selected each individual word, then rank the words to identify the most frequently selected ones.
Report the top most-selected words (for example, ‘calm,’ ‘expensive,’
‘innovative,’ ‘fresh,’ and ‘intimidating’).
Use percentages rather than raw frequencies to report the number of
times each word was selected. (For example, you may report that 71% of
participants selected the word ‘fresh’ to describe the design.)
If you have multiple user groups and can identify those in your
participant responses, include them in the presentation of your
results. Meaningful differences between the sets of words preferred by
the two groups may give you insight into their different attitudes.
(For example, you may report that 54% of experienced users described
the design as ‘exciting’ while only 13% of novice users selected the
If you’re evaluating multiple designs or multiple versions of the same
design (for example, old and new), look at the differences between the
sets of words chosen to describe the different designs. (For example,
you may report that 83% of the users described the redesigned app as
‘professional,’ compared with only 20% using the same word for the
older version of the app.)
If the site is intended to communicate specific brand attributes,
decide in advance what words correspond to your brand positioning.
Then count how many users include at least one of those words in their
Use a Venn diagram to present how your results map to design direction
words, how different designs are described differently, or how
different user groups describe a design differently (see the example