We are developing a system which uses a web console. Once the user logs in, he can choose a date from a date slider and click the “Get” button to retrieve relevant data which can be more than 5000 records (The data that is retrieved is from the date that was chosen until “today”). For 5000+ records, it takes us ~5 minutes to bring the whole data so while the data is loading, the user can only read the already retrieved records without able to search \ edit \ filter.
Users can use the records to get an overall state about their system status or for investigating past events within a timeframe.
My question is: Should we allow the user to create a custom query before clicking the “Get” button and minimize the data\records retrieved from the DB (we lose the “live” search feature + show limited records based on the query) or leave it As-Is and just highlight the Loading and make the user wait until all the records are done retrieving (which will allow a search all over the data in “live”). What do you think users will prefer ?
Note: we do paging for the data\records (but still, only reading)
Thank you !