I’ve no data other than some personal observations, but analytically modal lightboxes have such major usability problems compared the alternative that I can’t imagine what they’d be good for.
People have compared lightboxes to modal dialog boxes, which force the user to complete or cancel an act before proceeding, while still allowing users to see the previous content for reference. Assuming it’s actually a good idea to force the users to do something (and usually it isn’t), this forgets that dialog boxes have a major advantage over lightboxes: they’re fully moveable. The lightboxes I’ve seen can’t be moved, so the user can’t see all that’s behind for reference. Even if a lightbox were movable, you wouldn’t be able to move it outside the window frame so it is always occluding something.
If you need something modal in a web site, move to a new page. Like a modal dialog box, when the user moves to a new page, the user is forced either to complete what’s necessary or to cancel (via Back). Nielsen may talk about how well a lightbox captures user attention, but it can’t compete with presenting the user an entirely new page. If you need context from the previous page, copy the appropriate content to the new page.
Compared to a new page, lightboxes add mental workload, forcing the user to learn and remember yet another way of navigating. It’s bad enough that there are three inconsistent ways of dismissing some content in order to return to previous content. The user may need to use the Back button, or close the current window, or close the current tab. Now we add closing the lightbox, with its own idiosyncratic controls. Users don’t need this complexity.
Worse, it breaks the Back Button. If you leave the Back button enabled, users may click it intending to back out of the lightbox to return to the underlying page (or to move back a page within the lightbox such as when viewing a slide show). But instead, clicking Back moves the underlying page back, forcing the user to go forward and re-set everything up. If you disable the Back button, users may take it to mean the lightbox content is opened in a new tab or window, so they click the window/tab Close button to go back, blowing away the underlying page, its history, and maybe the entire browser session.
Lightboxes don’t look like popup windows, so they used to be a way to get around users dismissing new windows as advertisements, but now advertisers are using lightbox-like elements, so that advantage is fading if not already loss. Besides, new windows are modeless, so they’re for a different situation than the modal lightboxes we’re talking about here.
In summary, lightboxes represent a poor re-invention the dialog box with no advantage and new improved disadvantages.