I'm working on a feature that will allow administrators to manage lists of resources (people, vehicles) that number in the thousands. One client even has over a million entries.

Making a simple list with pagination and sorting seems insufficient. I don't see users browsing a 1000+ page list like this. A basic search function would be a good first step, but I'm wondering if anyone would have tips, or online references, for managing gigantic lists like this.

EDIT: We do have a query builder in another part of the system, so it may be a good idea the bring it in. I'm in discussions with the developers right now to see what kind of data will be available for individual entries. So far the outlook is grim; they only contain the person's name and unique ID.

  • Have you thought of offering a query builder? Depending on your data architecture, it might not even be optimal performance-wise to provide all results in a list. EDIT: Sorry, hit enter. If you could adopt some kind of natural language filtering, you might be able to match your users' mental model. For example, it might be easier for me to type "Trucks used in the last 2 weeks" than to adjust two filter boxes to achieve the same result. Jun 9, 2015 at 14:17
  • What kind of filters are you using?
    – Mayo
    Jun 9, 2015 at 14:48

4 Answers 4


If the user has that large number of entries, I think is a better idea to sit the final user and get the requirements of the task that he is actually trying to do. I don't have all the context but what is he trying to achieve:

  • If the user has to review all entries one by one (for example an accountant needs to review all the balances to see what is not matching). If this case, then try to make the interface to make the user focus on what is revelant in the columns.
  • If the user is looking for something specific then filters is the way to go. Only show him the entries that he's looking for. The easiest for the customer to filter the better.
  • If the user making a periodic know bulk change (if it's a repetitive task sounds like you could create an automatic task and just give the user a report with how the task was done).
  • You have a point, I should spend more time on this before moving on to solutions. I'll do some research and come back if I have more questions. Thanks!
    – Frank
    Jun 9, 2015 at 15:26

Managing a list of anything... that is over ~50 items is tricky unless you have (or in this case provide) the right tools.

  1. Paginate the results - seeing more than ~100 of something on a screen is not very usable (If the user needs to see a larger set/all at once, provide an export option where they can manipulate the data externally)
  2. Sorting - provide sorting on the column(s) that make sense, and make it easy to sort (typically clicking anywhere on the "header" of a column should allow sorting ascending/descending)
  3. Filtering - with filtering you provide the end user to see the data they actually care about (e.g. just SUV vehicles, just leases, just Ford vehicles)

With the above 3 items a user should be able to find everything they need.

Note: The quality of filtering matters too. Where possible keep these things in mind.

  • Date columns are best filtered on a date range (at minimum provide 2 text boxes to enter after/before dates, but better yet provide a GUI calendar they can choose from, and/or provide built in common filters (e.g. last 7 days, last 30 days, Q1, etc.)
  • Text columns are best filtered with a wildcard text input (e.g. "V102" when searching a VIN# column) would return all results starting (or containing) "V102" (starting/containing will be up to you depending on performance needs)
  • Hierarchy columns are best filtered with an open to pick "higher level" condition. e.g. If you have an "Employment Status" column for staff let the user choose "Employed" as a high level status which covers "Full Time", "Contract", "Co-Op", "Temporary", etc.
  • Defaults should be pre-set to best aid the user in finding what they are looking for. e.g. when searching for employees you'll likely want to pre-filter out the non-active (retired, fired, laid off, etc.)

Basic requirements

I'd second everything scunliffe said. Sort and filter are critical. Pagination is once you get past 100 or 1000, depending on the data and speed of your service. You mentioned supporting search and query-building, but it deserves repeating that simple search is a given.

On the point of pagination, it might help to think of it in database query terms: Is the user likely to want the top 20, 100, 1000. Base your pagination on that number. For bonus points, allow the user to select their preference.

A diffent animal ... sorta

A lot of experimentation and testing has been done in the ecomm space with this problem. This space is a good reality check for what average users are accustomed to in their "normal" life. Zappos has always done well with this, and backed it with a lot of A/B and MV testing. Here's their current sort and filter tools.

Zappos' sort

Zappos' filters

A pattern you'll see at Zappos and many others is the ability to deselect filters in their expected location (the filter UI) and in a "current selections" area at the top of the page (seen under the results count here). This is helpful when the the filter UI might be hidden due to the depth of options or contextual activation.

Power search

The functionality behind filtering and date selection (just another filter) is critical. But the UI for it can take many shapes. At the minimum you need a visual tool (e.g. a query builder) for the average user.

But if you'll have power users who use this on a daily basis to perform complex look ups, you'll need something more powerful and efficient. This is where the "omni bar" or "smart search" pattern shines.

Dream with me for a moment ... [ dream sequence soundtrack ]

enter image description here

Notice I tossed in a little syntax highlighting while I was dreaming. The engineers will love you.

This may seem like an edge case, but if you have power users (the type that will learn your key sequences*) they will sing your praises for this. They will become ambassadors and educators among their peers. They will relish in their ability to perform wild and crazy stunts to divine insights to answer every executive question.

* You have key sequences, right?


Always start by asking what users need to do with the data

Do they need to go into a specific record to look up info or make edits or bulk edits?

Put in easy search & filtering. Determine what are the general context users start with when it comes to finding data. Do they do a specific search (e.g. name, email, phone #) or by ranges? (e.g. by city, area codes, gender, age). Make sure your basic search functionality allows the user to readily perform searches of interest. If specific ranges are common search parameters, provide these as facets in a faceted search.

Question to ask: Are the users more likely to try to drill down to find their specific record or are they more likely to spiral out to find related records? "AND" Filters tend to work better for the former case and "OR" facets for the latter.

Or Do they need to survey their dataset and then perform actions on selected few?

In this case, it makes more sense to show them aggregated summary on their data in form of a dashboard. Then allow them the ability to drill down into specific records. This is more typical for admin and manager type users who needs to oversee things. The dashboard will have pre-defined target metrics put in place and can warn the user when something/somebody is under-performing.

If this is your use case, you need to speak more with your users to determine what are the metrics that matters and design the dashboard based on them. You'll also pair this with a simple record search for the times they do need to look at the details of a specific item.

Now onto the fun decisions

Pagination vs lazy loading

Are pages of results meaningful for the user? If this is a result set that the user will reference in the future, then yes pagination is helpful. If the current search result set is only relevant right now. (e.g. twitter feed list) then lazy loading makes more sense.

Advanced Search

Example from JIRA search

enter image description here

Having a natural query language is a good idea for advanced/power users... especially if you need bulk edit functionalities. Keep in mind that regular users typically get intimidated by seeing an empty query box. Syntax aren't hard to learn, but it does need to be learnt. So always provide a simple search option with the advanced query option.

Also the dev efforts to put in natural query language isn't small... so unless your user base need to perform routine complex searches, you may not need this.

Save Searches / Sensible Defaults

Are users consistently conducting similar type of searches in the system? You might want to have a default set of search results load when the user first arrive on the page. If this is more of a general usage type of system, a default search set might make the most sense... e.g. list of most recently added entries. Sometimes searches may be user specific... in that case, it might be helpful if the system recalls their last search parameters and display that to them. If this is a more advanced internal tool, the ability to save search queries may be useful.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.