If we ask UX-guru Jakob Nielsen it's 10 seconds. Longer waiting times could get the user to leave the program/page and do other stuff in the meantime. Supposing that something has gone wrong also depends on the users anticipation on how long a certain task could possibly take and the kind of task itself.
Taken fromOriginal (1993): http://www.nngroup.com/articles/response-times-3-important-limits/
Revised Version (2010), taken from http://www.nngroup.com/articles/website-response-times/
The 3 response-time limits are the same today as when I wrote about them in 1993 (based on 40-year-old research by human factors pioneers):
0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.
0.1 seconds gives the feeling of instantaneous response — that is, the outcome feels like it was caused by the user, not the computer. This level of responsiveness is essential to support the feeling of direct manipulation (direct manipulation is one of the key GUI techniques to increase user engagement and control — for more about it, see our Human Computer Interaction (HCI) for Real World Problems seminar).1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data.
1 second keeps the user's flow of thought seamless . Users can sense a delay, and thus know the computer is generating the
outcome, but they still feel in control of the overall experience and that they're moving freely rather than waiting on the computer. This
degree of responsiveness is needed for good navigation . 10 seconds
keeps the user's attention . From 1–10 seconds, users definitely feel at the mercy of the computer and wish it was faster, but they can
handle it.10 seconds is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.
After 10 seconds, they start thinking about other things, making it harder to get their brains back on track once the computer
finally does respond. A 10-second delay will often make users leave a site immediately. And even if they stay, it's harder for them to understand what's going on, making it less likely that they'll succeed in any difficult tasks.
Even a few seconds' delay is enough to create an unpleasant user experience. Users are no longer in control, and they're consciously annoyed by having to wait for the computer. Thus, with repeated short delays, users will give up unless they're extremely committed to completing the task.