How to present a huge list that support searching and scrolling in a fast way?

Question

We are using jQuery UI Tabs to display 2 tabs with huge data; each tabs contain more than 8,000 drugs.

The tab switching is currently extremely slow, it takes 4 to 5 seconds (on Chrome, IE) and 2-3 seconds on FF. It's not user-friendly even with loading indicator.

So my question is: Is there any other way to make the UI more user-friendly and the tab switching faster ?

I was thinking of something similar to infinity scroll (lazy loading) but there is a challenge: user should be able to type and scroll to any item that match the search term. To do that, it requires the whole list to be present at page load.

PS: the original requirement is to let user access the drugs list easily. Our approach is to let user scroll through the drug list, and type to search. Any other, better, way to present the huge list to user?

Our current implementation:

Answer 1

I can't imagine any scenario where a user would benefit from scrolling manually through a list of 8000 items. As such, I don't think a list is the best default UI element to use here.

I think you're halfway there with the search field. I'd show that as the default view:

[ (type term to search)             ]

Then, upon a minimum threshold (say 3 characters) do an auto-query to the server to start creating the auto-complete list:

[ asp|                              ]

  - aspirin                        /\
  - asplorem ipsum                 ||
  - aspdolor                         
  - set aspaquit                   \/

This forces them to start with a sub-set of the total data and, IMHO, is a requirement here as it's simply unfeasible to scroll through 8000 items.

An alternative option--or perhaps one that can be used in combination is an alpha-index:

[A] B  C  D  E  etc...

 - Aspirin
 - Another thing beginning with A
 - Another thing beginning with A
 - Another thing beginning with A

But note that that would still be, on average, 300 items per list which is still a bit daunting.

Answer 2

Don't load all the data upfront like this. Send your filter to the server to request results, but don't load them all at once. I would present the top 20 or so (depending on how much screen space you want to devote to this) results from the set in the drug list, essentially around double what will be visible to the user without scrolling. Then write some javascript to fetch more, triggered only when the user scrolls. This way the user can scroll if they want to see more, but you are not loading any unnecessary data.

It is likely that the user will never scroll through all 8000, but it may be annoying if you limit the results set to X when the record they want is at X+10 but they can't exactly remember how to spell the whole drug name (they just know it when they see it).