Skip to main content

# Tag Info

## Hot answers tagged statistics

22 votes
Accepted

### How many test participants to 'prove' an issue?

You asked if there is any statistics that can be used to essentially determine if the 10 people you are testing with are especially dumb. Essentially what you're asking is: What is the likelihood ...
• 344
20 votes

### A/B testing - client wanted the test run 70/30

Your statistical power is maximized if equal numbers of users are directed to A and B. That is, assuming one design really is better than the other, you’ll achieve statistical significance with the ...
18 votes

### How many test participants to 'prove' an issue?

In one study I discovered "5 out of 5 users couldn't complete the task" in the past, and my stakeholders simply didn't believe the data. So this is a something to consider. While Nielsen advocates 5 ...
• 8,934
17 votes

### Percentage of screen readers users in USA?

To answer the askers question, it looks like we have to use a little deduction. Nobody seems to collect these statistics outright. Someone wrote a very detailed article as to why. There are 326 ...
• 271
4 votes

### Confidence intervals, Confidence levels and probability of simple tests

From your example, let's imagine you have run the fake door test, which resulted in 265 visitors clicking on it, where total visitors that day were 350.   The real question (as you rightfully ...
4 votes

### A/B testing - how to deal with minority that chose B?

'C'. Option C should be a working model that introduces the compromised hybrid solution that attempts to find a way to appease both testing groups to a larger percentage. It'll always be changing, ...
4 votes

### How many test participants to 'prove' an issue?

From a statistical perspective, the larger your test group, the more accurate your result. With an increasing number of participants, your result in the limit value always approaches the real value. ...
4 votes

### A/B testing - client wanted the test run 70/30

Run A/B test on 60% of the traffic then. Diverged exactly 50/50 into old and new variants. Diverge all other 40% of users/sessions/views to new experience (as your client requests), but ignore them in ...
• 141
4 votes

### A/B testing - client wanted the test run 70/30

In theory, there isn't a problem with a client-ordered 70/30 test, as long as you're comparing conversion rates and not conversion amounts. If your hypothesis holds up, you'll still see the 5% dip ...
• 12.7k
3 votes

### How can i interpret the observed difference in a/b test?

You are correct that a significant null hypothesis result only tells you that A and B are different, but it doesn't quite tell you how different they could reasonably be, given the sample size and ...
3 votes

### What percentage of internet users save credentials in the browser?

Here's an article from Pew Research that points out which method is used by users to save their credentials from a survey in 2016. [W]hile 18% say that they save them using the built-in password ...
2 votes
Accepted

### Best way to approach statistics pages

Say for example, there are around 10 numerical stats that we want to show. How should hierarchy be determined when we are unsure of what data is most important? This must come from business. ...
• 6,641
2 votes
Accepted

### Graphs comparison placement

Having one before the other might be inevitable. You need to think what is more important to you. Here are a few questions that might make you choose one representation over another: What separation ...
• 481
2 votes

### How many test participants to 'prove' an issue?

Most usability questions can be answered with small batch, qualitative testing, as mentioned above. Some questions require large samples and quantitative proof to answer. A/B testing uses large ...
• 1,549
2 votes

### A/B testing - client wanted the test run 70/30

Your customer wants two things: Find out with evidence which method gives better sales, and don't lose sales during the test. A 50/50 split will give you the evidence quickest. A 70/30 split ...
• 486
2 votes

### How many test participants to 'prove' an issue?

Change the way you communicate with stakeholders If you need evidence to resolve a deadlock between stakeholders, the battle is already lost. Good design is not about quantitative science, it's about ...
• 4,813
1 vote

### What percent of the web's images have alt text?

A couple top hits searching DuckDuckGo for: accessibility statistics "alternative text" ...recently report approx. 75% of "home pages" have alternative text for images. Sources ...
• 2,653
1 vote
Accepted

### How much does bounce rate vary according to page load speed?

According to Pingdom, the conventional wisdom has been more like two seconds (Google's recommendation), whereas the average website takes 3.21. The drop-off is much steeper than you suppose, based on ...
• 1,121
1 vote

### Any best practice on how to represent player progress stat in a game?

It seems like for a particular 'exercise' or 'task' you want to represent two bits of information. Progress of the exercise (i.e. how much of it they have completed) Competency of the exercise (i.e. ...
• 27.3k
1 vote

### Are there stats about height of browser windows worldwide / by industry?

I personally use this website call statcounter. I find it quite useful to decide with the team what viewports we should support, what breakpoint we should take into account, and what's the smallest ...
• 11
1 vote

### How to display data with variation to unsophisticated users?

Did you consider giving median with a label very uneven (red) : std > 4 uneven (orange) : std > 2 homogeneous (green) : std <= 2 Sometimes if you draw the statistical distribution, it shows ...
• 51
1 vote

### How do I treat an 'Incomplete' in a Usability study?

I would mark them as "Incomplete" and write down the reason and details for not being able to complete the task. I also do this for tasks where the facilitator had to step in and help the participant ...
• 1,412
1 vote

### Confidence intervals, Confidence levels and probability of simple tests

The test that you described is not a true experiment or test. The hypothesis is unclear, the results cannot be reproduced or generalized. The method is more of a user survey, so there's no point in ...
• 1,412
1 vote

### What is the best way to statistically analyze a score system and number of steps for users?

Your metrics probably cannot be easily aggregated into one number. One is subjective (ranking) the other objective (number of steps). One way to summarize both could be to use a grading system (A+, B,...
• 698
1 vote

### What is the best way to statistically analyze a score system and number of steps for users?

John Brooke's SUS provides a kind of quantitative evaluation and final number/score that could be customized for doing task analysis. Template can be found here: https://www.usability.gov/how-to-and-...
• 485
1 vote

### What to do when sample size is too small during usability study?

Calculated with the Adjusted Wald Method, as N goes up the confidence intervals shrink. Are you evaluating specific tasks as pass/fail, or doing something more involved? Are you comparing something ...
• 1,537
1 vote
Accepted

### Should I invite existing users to my summative usability test?

Would inviting existing users hurt the results of my summative usability test too much? Yes Should I rather invite existing users for a separate test? Yes Is there a benefit of including ...
• 38.1k
1 vote

### What's the best way to interpret Likert Scales with equal negative and positive Likert Items?

I would suggest that you might be able to present it on a single scale as "perceived data accuracy", and then just reverse the values of the negative ones and combine them with the positive ones. ...
• 3,501
1 vote

### Some stats about social networks used by iOS/Android users?

As of 2016 the number of US users of the Facebook app has reached 47 million a month on iOS and 55 million on Android. Facebook is the most popular app on iOS and it is only number 3 on Android. ...
1 vote

### Are there devices narrower than 320px (and data on their usage for web browsing)?

According to yesviz.com/devices.php, each of these devices has a viewport width of 162 px: Apple Watch SE, 40mm Apple Watch Series 6, 40mm Apple Watch Series 5, 40mm Apple Watch Series 4, 40mm ...
• 121

Only top scored, non community-wiki answers of a minimum length are eligible