Sorting and pagination of unknown data set

Question

We are building a system that are dealing with a huge amount of information (mainly data presented in different tables). When fetching the data only 100 items is apparently fetched from the server, without knowing how many items there is available.

• How should we make this obvious for the user? "You are only viewing 100, but there might be more items available)

An ordinary pagination will be hard to accomplish, since we will not know how many pages there are available. We are considering this as some sort of "pagination"

Thoughts?

• Another question is how we should manage an ordinary sorting on table headers? Is it obvious that a sort on column A will only affect what you are currently viewing (the 100 items)?

Personally I would find it a bit strange if I have performed a search and Item A-K is displayed in the table, but I am aware of the fact that there exists an item Z. I think that I would do a sort on the table header and expect to get Item Z on the top (not Item K)..

Thoughts on that?

Answer 1

"Sortable results" and "partial display of results" do not mix, unless the sort actually does sort against the full data set (such that a reverse sort goes and gets item Z etc, rather than just sorting whatever subset of items happen to already have loaded.) A table column sort that only sorts the currently-visible items, rather than the full data set, is worse than useless, because it produces misleading results for the users, and will have tons of undefinable corner cases. Here's one:

• Do a text search for "frog", it goes to the server and retrieves 100 frogs starting with "Alpha Frogs" and ending with the "Garbage Frogs"
• Click the column header to reverse sort. The list now begins with "Garbage Frogs" (because it was sorting only the already-loaded items).
• Change the text search to "frogs". This goes to the server and retrieves 100 names starting with "Zzyzx Frogs".

In other words the same search / sort can produce completely different results depending on in what order the user happened to perform the search. That, needless to say, is bad design.

(I've also seen even worse implementations of this, where a new text search simply discards any existing sort settings, which "solves" the inconsistent-results problem by making the sort settings useless -- now there's no way to get to the Zzyzx frogs other than by clicking through the entire set of pages until you find the end.)

So: if you're going to have paginated results, every sort operation must result in a server call so that the sorted, paginated results accurately reflect the available data -- "Only search against whatever I happen to already be looking at", which is what your proposed column sort would do, is not something anyone would set out to design on purpose. (Plenty of sites have, of course, wound up 'designing' it by accident. Those sites are bad, and they should feel bad, as the saying goes.)

The "undefined number of pages" question is much easier to answer: just "Show more" (or equivalent text), which appends to the existing table rather than dropping already loaded results.

The utility of a "page number" field is quite limited: what use is skipping ahead to page n unless I already know exactly how many items are in your database? It's really only useful when stepping back to previous "pages", which is unnecessary if you append new results rather than blowing away old ones.)

Answer 2

I would put just one link below the table "Show next 100 records" and since sorting is usually done across the full data set, doing only current view sorting is not very common. I would put a note saying it only sorts current view of records..