As Gerda said, it's related to Gestalt, but there are some considerations (also based on Gestalt). When elements are closer together on one axis than the other, the reading changes. This is called the law of Proximity, but it also involves the law of Common Fate. > The Gestalt law of common fate states that humans perceive visual
elements that move in the same speed and/or direction as parts of a
single stimulus. A common example of this is a flock of birds. When
several birds fly in the same direction, we normally assume that they
belong to a single group.
Source
To better explain this, consider the following image

If you place the elements close enough to the horizontal axis, you'll read these elements from left to right.
However, if they're closer together on the vertical axis, read them from top to bottom.
Cultural considerations
The above is true for Western languages. However, the situation is quite different for East Asian languages. While there is no set standard in most languages (contrary to what most people believe, these languages are usually written from left to right these days), many of them use vertical writing. And when they do, it's preferred over reading horizontally, since each glyph represents a square space.
For example, the following image shows the same space for characters in both axes. Nevertheless, it must be read from top to bottom and then from right to left. You can see that western numerals are rotated to fit this orientation.

Although (as mentioned) it's becoming less common to keep the RTL alignment, it's important to remember that if the language has columns, regardless of spacing, the columns take precedence over the lines.
Finally, we have the Arabic languages. With some specific exceptions due to design, readers of Arabic languages will always prefer rows over columns. There are many considerations on this topic, so I'll leave you with theW3C Text Layout for Arabic Languages page.