Handling large lists of comments on multiple items within a page

Question

I have a site that includes lists of 25 items, each one of which has its own comments block, and each comment can be linked to directly, in which case the page is loaded, scrolled to that comment.

Previously, the comments worked as follows: we'd show first comment, a "show more", then the two most recent comments. Show more would expand all of the intervening comments. If someone linked directly to a comment that would be hidden, we loaded the page with that comment block expanded.

The problem with this system was that popular items could end up with thousands of comments, and page loads would time out or run out of memory.

We replaced the old comment system with paged comment blocks, 5 comments per page, paging in place on the list page. If someone jumped to a comment in the middle, we'd load with that comment showing. Otherwise, we load the last page, with the most recent comments, for each item, which may have 1-5 comments.

This solves the problem with page loads crashing, but it is not popular with the users who were used to being able to see all comments at once, without paging. Is there a better pattern for handling showing/paging of comment streams?

EDIT: I should have mentioned this previously: comments often reply to one another, so the sequence of comments is important. We don't currently store threading data, so ordering by top comment would cause issues with following conversations.

Answer 1

Paging in general is the best way to go and additional content should be loaded as needed. However, when it comes to comments, you can often have problems that are a result of improper ordering and you've mentioned a few.

The best way to go about ordering and scaling comments is by assigning a popularity to the comment made. The popularity could be the number of replies to the comment, the amount of up votes to a comment or any other similar mechanism.

Once you have a score for the comments, when pulling it from the database you should be able to pull the comments in the right order based on popularity. You could also add an option to order by most recent.

As for loading I would suggest you employ an infinite load. You should automatically load more comments as the user scrolls down to save them from clicking "Next Page" or whatever you have setup. Ensure you add a limit to that, something like after the 5th page force the user to click "Load More" so that they can load up to 5 more pages.

Answer 2

Consider the purpose of those comments.

It is either a way for a user to ensure validity of the rating or to find additional information (pros/cons of the item).

In both ways, rating and 3 top comments would be totally enough to shift decision scale of a user to one or another item.

Displaying additional comments should require certain user initiation, say, hovering mouse over the item, just because of typical user cognitive limits.

Also it would be reasonable to scale number of comments with layout of items positioning, for example, if you have 2 display modes, item-per-row and iterm-per-cell (table layout):

1. When user choose item-per-row it means he is 'meticulous' type and interested in top 10 comments. So you have space for those comments and user want them - it looks right to show them.

2. When user choose item-per-cell - it means user want quickly overview all items. He may need minimal item feedback, rating and maybe 1-3 top comments (in some cases it would be advisable to not show comments at all).