2

The file contains thousands of PDFs and will be posted on a government website. The primary audience is researchers and the media, desktop, not mobile. We plan to indicate the filesize. Given 1.2 gb is pretty big, are there any reasons we should split the file into smaller parts? Asking because I'm just not sure.

  • Is it necessary that all files be downloaded together? Or is it just a case where the client is lazy to not create a proper library and dump everything together in a compressed folder? Are you asking about the technical restrictions of downloads? – Ren Aug 6 at 19:38
3

I think you should split the file into smaller parts, but not necessarily by file type but how the user consume them.

Lets say your file contains documentation and media files for Java. PDF, Video, Images , and Code file all make up to 1.2 GB. You might wanna provide them as:

  • Table of Contents
  • Core Java
  • Advanced Java
  • Java books
  • Java Tutorial Series
  • Download all - 20 minutes to download based upon 1mbps connection

All the items should be individually downloadable but user should be able to download it as package.

2

I occasionally use large datasets for analysis and appreciate being able to download everything in one go. There are, however, a couple of points which helps usability a lot:

  1. Check that pause and resume download works in chrome/safari (it will usually just work on a normal web-server or CDN).

  2. Provide a plain HTML-link to the file without authentication. Allow the user to download using alternative tools such as WGET or Curl easily.

  3. Don´t do cache-busting (Web servers are good at handling static files and won´t mess this up by default).

A few gigabytes is a non-issue for most people in 2019.

Cheers,

1

I've worked on .GOV and GDS.

You should start by discovering what the user needs are.

"As a user I need all the thousands of files because of X" as opposed to "as a user I need access to thousands of files I can search and browse as a resource" are very different and will greatly affect your solution.

My gut feeling is 1GB file (zip) of all PDFs is a total no-no.

But do your research.

  • I wouldn't class it as an outright no-no; I've encountered situations where I've wanted to download what amounted to several GB of documents. However, in every case where that was a possibility, the documents were stored uncompressed, and I added the ones I wanted to a 'basket', which was then compressed on-the-fly for download. – Jeff Zeitlin Aug 6 at 15:04
  • Without going into too much detail, we have done the research and there is a use case for providing a compressed file with all the PDFs. But, my gut feeling is similar to yours in terms of the file size, and I'd like some thoughts around what considerations we should take into account. – Yoona Koo Aug 6 at 15:10
  • then you can have an option to "download all files" but it's very much a secondary option. Your interface should be geared around display a list of PDFs the user can pick and choose from. – colmcq Aug 7 at 10:23
1

Provide both an all inclusive download and smaller organized sets.

Many users may decide to download the whole thing, but it is not unimaginable that someone may need only a few files from such a large dataset.

For example, what if Alice mentions to Bob that she is looking for statistics on X event. Bob remembers that those figures can be found in a set of 50 PDFs from your download. Without finer grain control, Alice now has to wait for thousands of files to download just to get the 50 she needs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.