I don't particularly like it. And the reason has to do with the visual separation of questions on a page. In short, the voting buttons are presently the most distinguishing feature identifying the top of a question and act as a very intuitive visual separator between questions. I found that when the voting buttons scrolled with the top of the window, I got confused trying to quickly navigate between questions because my first reaction felt like the questions themselves were moving around on the page.
The moving buttons add a whole new dimension of visual confusion that the brain has to process. It's an unnatural behavior the user has to be conscious of. That is, you simply have to be conscious and aware that the voting buttons are moving as you scroll or you will get confused very quickly while navigating.
Furthermore, by anchoring the voting buttons to the top of the window, the buttons for the current question will remain in roughly the same place on the screen the entire time (only to change briefly as a new question is scrolled in). This also makes it much less obvious when you've scrolled into a new question, unless you happen to be closely watching the voting buttons the whole time.
The human brain is very good at spatial visualization. By keeping with the "page" metaphor, where things are fixed on a single scrolling plane, the brain can visualize (to a degree) the entire document above and below the viewport as you scroll. But when the document changes as you move it, that impedes the natural spatial association of the document as a whole.
Just observe the animated GIF. It becomes very difficult to visualize the entire page as it scrolls, and to effortlessly discern the boundaries between questions.
I would argue that if the designers really want to do things this way, that more is needed than the current trial implementation. In particular, the voting buttons need to be conceptually separated from the post body (perhaps by having a darker background color in that column), and there needs to be some improvement made to the visual separation of questions.
For example, even something as simple as this could have a huge impact on the readability of the page with floating vote buttons:

Now even though the buttons are moving up and down within their column as the page is scrolled, the containing columns themselves are static, and since they are visually dominant, the brain has an easier time keeping the answers separated as the page is scrolled.
In short, the solution is to make the voting buttons less visually dominant. But being less dominant doesn't mean less obvious mind you. They're still obvious, but because they are contained in a visually dominating rectangle, the brain can tie that dominant rectangle to the page, and movement within that rectangle is not as distracting to the scrolling as a whole.