it's a very bad idea. First of all, if you happen to have a vertical scroll, what do you think will happen if all fields were horizontal? Unless user has a very wide screen, chances are that in 90-95% of cases user will have to scroll horizontally (think on this: more than 50% of your users will see your form on a mobile). And horizontal scrolling is really unnatural for users, while vertical scroll is what user expects.
However, despite what you may see here and there, you can have 2 columns. In this page, you'll find some sound advice about form building, but I don't completely agree with the number of columns part. I'll explain: For reasons beyond my control, I had to build a form with 3 columns once, and user testing was really bad. However, I have built many 2 columns forms and they work almost as well as a single column form, only that saving space (but make not mistake: if you can do it in ONE column, go for it). The problem with that example is that they consider a form in 2 columns will be built in an inverted-N shape, which makes no sense. Forms in 2 columns have to be built using a zig-zag pattern, which also explains why 3 or more columns are really confusing.
What about labels?
Another thing to consider is labels. If you use top-aligned labels, you'll be able to use 2 columns without issues in most cases, from desktop down to tablets. However, if you use inline aligned labels, chances are you'll only be able to use 2 columns on desktop. Maybe tablets, depending on label and input size and font used. But this is something to consider nevertheless
The strongest case against horizontal forms is...
textarea. Just imagine having a number of
input fields taking around 30px height and then a
textarea with 100px height. It will break any layout without any question. Same will happen if you have several
radio inputs or
checkboxes, your form will get extremely long, requiring a really weird horizontal scroll. Just imagine having 5-6 options and then scroll to the right until you lose focus on the label, or even worse, all other fields. And if you list the options vertically... you'll break the layout, same as with
And I could go for hours explaining why is this wrong, but guess you got the idea by now. So, as I said at the beginning: it's a really bad idea