Horizontal scrolling like all other features requires certain conditions to exist in order to be comfortable. These conditions include display technology (screen & layout) and the navigation control technology (input devices).
Let's look at the brief history of them all.
Nearly all analog means of presenting information have had horizontal scrolling: be it old-school parchment scrolls or bound books ("pages" were turned along the height not width). (I think there also were vertical scrolls but I couldn't find any photos of them.)
When we jump to the digital age, we find that the video chips of early computers had rather tiny resolutions (e.g., Commodore 64 had just 320 × 200 pixels, which translated into barely 10 lines per screen). The second peculiarity of early computers (I've skipped the punch-card age) was the lack of a GUI, which meant that input commands and output where all typed and presented into the same screen.
I guess the thinking of the original CLI designers was along these lines:
When we reach the end of the line during input, we push the entered text up and continue typing on the new line. When the screen fills up with text, let's just keep adding more lines as we need. This way we'll have an infinitely tall sheet of paper that we just need to scroll up or down somehow. Awesome!
Interestingly, the aforementioned Commodore 64 didn't even have dedicated page up/page down keys (according to some manuals, they were coupled with the F-keys on the right).
Jumping another 10-15 yeas into the future to late 1980s and the first GUIs, we see computers conquering their place in everyday office life with spreadsheets and rich text editors, the latter being the modern reason for vertical scrolling. The low resolutions (even with IBM's high-end 640 × 480 VGA in 1987) couldn't display one letter-sized page (or A4 in Europe) per full screen. There was no other way but to scroll vertically to read one page. To remedy the situation, Microsoft invented their zoom/scroll wheel in 1993-1994 and Apple patented Mighty Mouse with thumb scrolling in 1992. The mouse controls were obviously made to follow the pre-existing scrolling paradigms.
Even today, vertical resolution isn't high enough to fit a letter page despite allowing 2 pages horizontally. This is a screenshot of Microsoft Word 2007 on a 1920 × 1080 (save for the Windows 7 taskbar):
The Internet as we know it was born in early 1990s and had to follow the conventions & paradigms of desktop applications, which weren't easy to defy when the keyboard had Page Up/Page Down keys. The conventions influenced not only design guidelines but also the architecture of HTML rendering: DOM elements are placed to the right of the previous element as long as there's unused space and then shifted to the next line below it increasing page height. Yet, even vertical scrolling on the Web wasn't recommended until 1997.
In the recent years a few important developments have happened, which signal a potential shift toward horizontal scrolling in the future:
- Some mice now include "tilt" wheels for horizontal scrolling but they still aren't common;
- Wide screens have become popular and are standard on nearly all laptops and most desktops;
- Multi-column layout controls were added to CSS3;
- Mobile devices (iOS, Android, and Windows Phone) have started using horizontal swiping gestures extensively for navigation.
However, horizontal scrolling is still far from adoption on full-sized screens. A study presented at WWW2009 (PDF) found that users preferred vertical scrolling because they were used to it as opposed to horizontal scrolling and some pointed out that horizontal scrolling because it "forced them to move their eyes vertically up and down the full height of the screen", which isn't the case with vertical scrolling. The latter reason is a strong indicator of ergonomic preference for vertical scrolling.