It very much depends on content. The optimal amount by which to display is just enough but no more than necessary.
But what defines 'just enough' and just enough for what.
Just enough to be able to relate the content on the next page to that of the previous page.
Just enough to be able to locate the next exact point of interest that follows on from the previous point of interest.
Just enough to visually confirm that if you were mid paragraph, there is a little bit of what you were just viewing that was the same content and that you are indeed at a point which is related to the end of the previous view.
Just enough to be able to see the whole of something without having to scroll or page back up to see the rest of it.
So this is in a way related to the chunking of information on a page - and the flow of information on a page.
If you're reading text, then it should not be so much overlap that you have to seek a long way down the text on the page to find the continuation point. So for example a couple of lines should suffice so that the context and 'just read' lines can be easily omitted. Less than that and you cannot determine that you really are where you left off.
It's important to have that transition content there to tie the two pages together. You see this on maps (especially real paper maps) where there is just enough overlap that if you were following a road or terrain as you go off the edge of one page, you can recognise a small section of it on the relevant edge of another page.
The brain is very good at building and recognizing patterns, and when you move from one page to another, you are not typically re-reading the text that you just read, but you are looking for that pattern of words you just scanned so as to be able to limit the flow of reading. Too much information and the flow is interrupted because you don't recognize the pattern. Since we read text one line at a time without having to follow with your finger from the end of one line to the beginning of the next then we're very used to chunking the lines. And that means we should overlap at the minimum by one complete line of text.
If we're looking at images, then clearly it could be useful to detect this and page down such that we see the next whole image or block rather than having it cut off at the top.
However - that's not the whole story, because we don't necessarily read text right to the bottom of a page before paging down. We actually are looking quite a lot further down the page than at first we think. we like to see a whole paragraph on the page so that we can process what is clearly a whole chunk without interruption. We also detect broken lines where we can only see the top half of a line of text at the bottom, and that's distracting too, so we might well scroll or page down in order to distance that from our current point of interest. Note that reading devices will never display halves of a line.
So you need to cater for those cases where we don't read to the bottom completely, which means that if we page down, we need quite a decent amount of overlap, and in fact 10% of a page may not appear odd in those situations.
It's a fine line, therefore, between overlapping enough to relocate a point of reference at the same granularity as our current chunking granularity, and overlapping so much that the point of reference we're looking for is too far down.
And that's where I came in - it depends on the type of content - but also much more subtle stuff such as the likely engagement of the reader; the likelihood of the user to be consuming the content sequentially or more randomly; the font sizes; the paragraph breaks; the visibility of the end of a paragraph or the completeness of the display of the bottom line (and the top line after paging).
Clearly, the page down mechanism is going to have to work hard to get it right all the time given the number of variables working together here, and it's a next to impossible task. So there's a middle ground which I'd suggest is somewhere between 2 and 6 lines of text depending on the relative height of the window to the screen.
These are just my considered thoughts on the matter - perhaps someone else will find a paper on it.
