0

Say I want to compare many machine-learning models based on the error produced by each model. In this case, smaller error means better model. If I want to visualize this, the first thing that comes to my mind is a bar plot. The result is something like:

bar plot

I feel that this plot doesn't express the meaning of the data. At first sight, you get the impression that "K-Nearest Neighbor" model is the best since it has the tallest bin.

So is there another type of graphs that expresses the data meaning better in this case?

7

Simple plain table does the job reasonably good:

enter image description here

Your plot is not perfect tool for comparison: it requires too much cognitive efforts to process.

  1. You need to get numbers from the plot. It's hard in the plot, but easy with table!
  2. You need to keep numbers in your head to compare these. Not good at all in the plot, but perfect in the table!
  3. Reading the models' name is hard in plot, opposite to a table
  4. Color usage is meaningless, it distracts

I also recommend Edward Tufte's books for more deep understanding the topic.

PS. I understand, the answer isn't about plotting (form) , but rather on your goal (function), which is

I want to compare many machine-learning models based on the error produced by each model

.


Update

And this view could work even better:
enter image description here

  • 1
    Additionally, you could add a second column with a % difference from the "best" option. K-Nearest would be 87% larger error than XGBoost, for example, (assuming I did my math right.) – mix3d Dec 10 '18 at 15:04
  • 1
    @mix3d true, I think they have some special term for this kind of metric, like MAPE (Mean absolute percentage error) or such – Alexey Kolchenko Dec 10 '18 at 15:08
  • 1
    I actually do not agree with most of this. To 1) that depends on how exact the number has to be for the viewer. If it's simply about better and worse the plot is easier to grasp. To 2) you don't need the numbers at all if all you want to do it see which one is better. 3) is true. 4) with the simple note that "less is better" it would make it clear that dark is worse. The bar chart enables you to quickly get a feel for the distribution. The table requires processing of the name and the number first. – Big_Chair Dec 10 '18 at 16:08
  • I think the table you put in your update is clear and simple. It resembles a bar chart with one color with the numbers placed on bars – ammar Dec 12 '18 at 2:59
  • I should clarify that I put a table before this chart in the report but I wanted to make the difference between models clearer for the reader, so I included the bar chart – ammar Dec 12 '18 at 3:01
3

If you MUST use a bar chart to convey this information, horizontal bar charts tend to do a better job of minimizing the "taller is better" interpretation. I would also use thinner bars, no wider than the height of your Font. This allows you to get the name of each model in a horizontal (read: easier to read) list, and the bar charts to the right of them. The current thickness (width) of the bars in the example chart take up far more space than necessary; thicker bars don't help to convey the MAE information, they just exist to take up horizontal space.

Example: Example chart

Secondly, the footnote of *Mean Absolute Error, smaller is better is a common approach to help charts of unintuitive number comparisons where smaller/lower is in fact, better. This is sometimes put ABOVE the graph, since readers will see that first as they scan the image.

@Alexey Kolchenko is right, the colors do nothing to help comprehension; if anything, they distract from the comparison. However, if you had several different graphs comparing these models and wanted to maintain consistency between graphs, assigning randomized primary/secondary colors to each model (so the relative color to neighbor items can't be interpreted as a data point) helps generate continuity between graphs.

  • @alexey I think the visual graph DOES help convey how much better XGBoost is compared to K-Nearest Neighbor (vs the table), as scanning 5-digit long numbers requires a larger cognitive load than simple shapes. – mix3d Dec 10 '18 at 15:33
  • 1
    Agree, but it depends on the goal. Such kind of plot is good for qualitative comparison, but not as good for quantitative one. Table provides minimal approach of solving both the goals, so I'd stay with it, unless the goal isn't 100% clear ). Also, table wins in ink/noise ratio. I liked you reasoning, though. – Alexey Kolchenko Dec 10 '18 at 15:56
  • 1
    I used a horizontal bar chart finally, but with colors. Colors don't add meaning (unless you consider the lighter the better), but I used them for an aesthetic matter. – ammar Dec 12 '18 at 3:05
  • I wasn't arguing for NO color, so much as showing that the shape was more important. If the aesthetics of wherever you're showing it means a pink bar fits the context better, go for it, but don't let color accidentally give more meaning than it should. – mix3d Dec 12 '18 at 14:57
3

Flip it upside down. Make the top of the graph the goal (zero error) and hang the bars underneath. This way you can still have the general idea of higher is better, and you can then chart going from bottom to top, following the standard visual layout for 'progress'.

Here's a random example from the internet: enter image description here

While this chart is about a single continuous statistic rather than several unrelated ones (performance of one algorithm doesn't depend on another algorithm) the visual layout still works.


Alternatively, if there is a maximum error, or if you can set an outer bound for it (for example, what is the error rate of a completely randomized outcome vs the algos) you can make the bars X minus the current value. For example if randomized is 50.000, nearest neighbour would be 27.000 better, and xgboost would be 37.750 better. This would mean that larger is indeed better, and you can use the current layout.


Consider mirroring left and right. Using the most important value first makes sense (especially if latter datapoints might be truncated!), sure, but it's in conflict with the 'improvement' metaphor of going to the topright. Having the worst first and the best last makes for a better 'story'.


I would also emphasize the good/bad with colors. Right now the colors don't have a clear meaning. Swapping either the orange or the dark purple for a medium/light blue should work fairly well.

  • I liked the idea of flipping the bar chart upside down; I will see if some Python package provide them – ammar Dec 12 '18 at 3:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.