Pop-ups are great. They're usually called modal dialogs because a user cannot do anything but complete the task in the popup. This is their primary functionality and they're better and clearer at that than most similar solutions. In OSX you also have what they call Sheets, but they're a lot less clear about the fact that you need to finish them before you can go back to the main application.
Windows GUI guidelines speak of modeless dialogs, ones that still allow you to interact with the rest of the interface. They would still be in front of it though, and probably in the way. Take for instance the Pop-over described in the OSX Human Interface Guidelines: you can not really interact with the calendar behind the pop-over. The nice thing these kinds of dialogs is that they offer other exits: you can still reach a menu item on the parent window, go there and at the same time close the pop-over. It skips having to specifically tell the pop-over to close and allows you to get on with your task more smoothly.
So, dialogs, even the modeless ones, interrupt whatever the user was doing, break out of the visual layout of the screen and totally grab the user's attention. If this is what you need to accomplish, they rock and there are many use-cases for them.
However, this incredible power tends to be overused. The primary problem is with the idea that you get to decide what the user should be doing right now. Clients, tend to love that sense of control. They'll feel that information X is super important right now and that the user should drop whatever he was doing and give that information. Programmers also love that kind of control so that they don't have to deal with not having certain information.
However, this totally conflicts with what is important to the user. People like to have a sense of autonomy and control, kinda just like your client does. Pop-ups take control away from the user, demanding attention to something specific.
Pop-ups also break out of the flow of the application and therefore cannot be part of that flow. If you're trying to build an interface where the user can focus on performing a task, you should avoid using pop-ups for parts of that task. That what makes them so powerful also makes them so awful: a user will need to reconfirm their position on the screen ("what is this, where am I") as soon as the dialog opens, and then again when it closes. Pop-ups by their definition break concentration.
A very important concept in UX design is the idea of flow as a state of mind. While this is usually discussed in the context of games, because concepts like goals and control are obviously part of that, these things are important in everything we do. We should strive to create experiences that make people feel great, whatever they're doing. Key factors for that to happen include the following (from the linked article, emphasis mine), and note how pop-ups work against a number of these factors:
The studies have suggested that the phenomenology of enjoyment has eight major components. When people reflect on how it feels when their experience is most positive, they mention at least one, and often all, of the following:
1. We confront tasks we have a chance of completing;
2. We must be able to concentrate on what we are doing;
3. The task has clear goals;
4. The task provides immediate feedback;
5. One acts with deep, but effortless involvement, that removes from awareness the worries and frustrations of everyday life;
6. One exercises a sense of control over their actions;
7. Concern for the self disappears, yet, paradoxically the sense of self emerges stronger after the flow experience is over; and
8. The sense of duration of time is altered.
tldr: use pop-ups when you need to, but they're not needed as much as you think and they get in the way of flow, making your users unhappy