How to figure out a threshold at which I should show "other" in a chart?

Question

I recently decided to add visualisations for one of my projects and found that there's too many data points to visualise them and not all of them matter. Since there's a lot of various data being visualised (as a pie chart, with an option to switch to a different chart), I decided that it might make sense to add "other" as an option. Now there's the question - how should I define a threshold at which "other" appears? Should it be more like a sum of all items below a specific percentage, should I be looking at the number of data points or would a combination of the two be most effective?

Example data might look like this:

Item  | Number of entries
-------------------------
Ex. 1 | 100
Ex. 2 |  50
Ex. 3 |  10
...   | ...
Ex.N-4|   1
Ex.N-3|   1
Ex.N-2|   1
Ex.N-1|   1
Ex. N |   1

In the context of the viewer, it's generally OK to sacrifice the Ex. N entries if necessary by grouping them together in order to highlight the main entries.

Answer 1

How are you differentiating the slices / data? I'm assuming (i.e. posh guess) that with a pie chart it's a colour based key. Standard advice above and beyond that relying solely on colour is bad is that people have a hard time differentiating beyond seven(+ or - 2) different colours. So that would lead to the top 6 choices being their own categories and the seventh being 'other'.

http://www.scribblelive.com/blog/2012/03/29/maximum-elements-for-visualization-types/

https://eagereyes.org/techniques/pie-charts

Answer 2

and found that there's too many data points to visualise them and not all of them matter.

Sorry, but I think you are mistaken, the outlier on the chart may very well be the one that steals the show or presents new avenues of thought.

Beyond self editing your data set for convenience , which again you do at your own risk, I would suggest you simply change to a different type of chart, a bubble or scatter might be a better fit.

In the context of the viewer, it's generally OK to sacrifice the Ex. N entries if necessary by grouping them together in order to highlight the main entries

I can only speak for myself ( as n= 1 viewer), but I want to see each one even if they are 1000 n-#s, I still think a bubble/scatter chart is your best bet, and again I'll try justifying my point: If one of your n-1 = 1 points grows over time, so it is now n-1=3 for instance, by grouping it you lose the ability to notice it.

You could of course choose to show 2 charts, the complete data set and an edited summarized chart with the most entries, to be accurate you should note your grouping/threshold scheme.

As for the threshold,I would group them by similar attribute, in this case number of entries.

One last thing, you might think I am pedantic, but in the history of displaying quantitative information (see Tufte) and in my own experience ( I used to prepare financial stock charts,research data for publication) this sort of details are the ones that end up causing mistakes.