Norman-Nielsen suggest 3 orders of magnitude in responsiveness:
0.1 second: Limit for users feeling that they are directly manipulating objects in the UI. For example, this is the limit from the time the user selects a column in a table until that column should highlight or otherwise give feedback that it's selected. Ideally, this would also be the response time for sorting the column — if so, users would feel that they are sorting the table. (As opposed to feeling that they are ordering the computer to do the sorting for them.)
1 second: Limit for users feeling that they are freely navigating the command space without having to unduly wait for the computer. A delay of 0.2–1.0 seconds does mean that users notice the delay and thus feel the computer is "working" on the command, as opposed to having the command be a direct effect of the users' actions. Example: If sorting a table according to the selected column can't be done in 0.1 seconds, it certainly has to be done in 1 second, or users will feel that the UI is sluggish and will lose the sense of "flow" in performing their task. For delays of more than 1 second, indicate to the user that the computer is working on the problem, for example by changing the shape of the cursor.
10 seconds: Limit for users keeping their attention on the task. Anything slower than 10 seconds needs a percent-done indicator as well as a clearly signposted way for the user to interrupt the operation. Assume that users will need to reorient themselves when they return to the UI after a delay of more than 10 seconds. Delays of longer than 10 seconds are only acceptable during natural breaks in the user's work, for example when switching tasks.
However, these are just rules of thumb, easy to remember but not particularly accurate. The goal is simply categorization of duration, so we can assign the correct type of UI feedback (nothing, spinner, progressbar, etcetera.). It is not intended as precise tool or measurement.
So, how long does it take, then?
What you're asking would be closer approximated by website bounce rate. That is, how many seconds do people let a page load before closing it deciding it is too slow.
Looking at just the first few charts on google, that time is drastically lower than 10 seconds:


On average these charts show the most significant curvature more around 6 seconds. What's also interesting to note that the first of these three shows an increased bounce rate after several years, though the timing of the bounces seems to be about the same.
Caveats Galore
Even though these charts are more precise, there are still a bunch of notes.
For one, when a page is loaded and when it is usable are two different things. If I'm clicking a link to a blog post, I can start reading before all images are loaded. If a site/app is just a static screen until it's 100% done loading, it will feel slower.
Then, there's expected duration versus actual duration; if I'm used to apps loading in 1 or 2 seconds on my phone, and yours takes 10 seconds, I will wonder if it started to hang. Some apps can get away with this better than others; if I'm starting a game, I can imagine it's a pretty big load, but if opening your camera-app takes several seconds it's not a good sign.
And of course there is a huge variety in devices and network speed. This is slightly mitigated by a user's previous experience with the device, but this doesn't account for things like our server being slow.
* ! RAMPANT SPECULATION AHEAD ! *
Perhaps 43 years ago the limit was actually 10 seconds, but a lot can happen in a lifetime.
U.S. men work 100 hours more per year than in 1970, women 230 more. The average commute has gone from 20 to 30 minutes in 40 years. We live in an on-demand society where we pay watch Dragons vs Zombies without ads, at our schedule. Our phones are orders upon orders of magnitudes faster than the fastest supercomputers of the 80's; the Cray- did 0.8 gflops for tens of millions of dollars while the iPhone 6 does 100+ gflops.
So maybe, in the almost 50 years since 1968, we have gotten a bit more impatient and 10 seconds have become 5...