On his article on website response times Jakob Nielsen points out that load times are not that big an issue anymore, response times are.
Today, most people have broadband, so you might think that download times are no longer a usability concern. And yes, actual image download is rarely an issue for today's wireline users (though images can still cause delays on mobile devices).
Still, response times are as relevant as ever. That's because responsiveness is a basic user interface design rule that's dictated by human needs, not by individual technologies. In a client usability study we just completed, for example, users complained that "it's being a little slow."
And then goes on to mention the classic 0.1s, 1s, 10s rules from Response Times: The 3 Important Limits.
But at the of the article is this little tidbit, emphasis added:
Response times are a matter of user experience: How much time does it take before the computer is ready to serve the user? The reasons behind delays don't matter to users. All they know is that they're getting poor service, which is annoying.