It is all relative. If it is a database application with millions of records a user will (should) have a different expectation.
My approach is for anything that might take longer than 1 second is to display an hour glass AFTER 1 second. I like to keep visual clutter down.
If it is a task that can take more than 10 seconds and you can display actual progress (e.g. 50%) then show them a true progress bar.
There are some search conditions that I know are going to be hard (long) and I give the user a warning up front this is going to be a longer search.
If it is a task that can take more than 10 seconds then give them a chance to cancel out. They may have entered the search wrong and it is taking a long time because of a weird search. If it is likely to take more than 10 seconds and there is no option to cancel out then give them a warning.
Give the user to option to automatically cancel out after X seconds. In my application if search is going to be hard they will often come at it from another direction.
Can you break the task up to return a count first. Like I have some bulk operations and I will return a count first and if it is > 1000 I tell the user are you sure you want to process X records? This will take a while. In a demo I saw a person mess up and create a workflow for then entire library of over 1,000,000 record and it finished in 10 minutes which is fast for that many records but it felt like an hour during the demo.
I have one common operation that takes 2-3 seconds every time. For that I don't display an hours glass.