Designing a web front end for launching long running batch processes

Question

I'm looking for an example of a "web portal" application which has a well-done, easy-to-learn user interface.

By "web portal", I mean a web interface to a batch-processed system that runs computational-intensive, long-running jobs. The typical workflow is something like:

1. Log into the web site.
2. Specify some parameters for the job.
3. Maybe upload one or more input files.
4. Kick off the job.
5. Check the status of the job (started? in progress? finished?).
6. Download the results of the completed job.

I'm looking for an exemplar: an example of an existing web application with a similar workflow that has a solid user interface, especially from the point of view of new users who have never seen the interface before.

Note: originally asked on Stack Overflow, but seems like a better fit for ui.

Answer 1

We do this very thing over at SmartyStreets. Users upload files with potentially millions of addresses in them and we verify and standardize/de-duplicate them, etc. It's very processor-intensive, and even with the master data loaded onto a RAM drive, millions of addresses will take a few hours (by the way, that's super-fast for that amount...).

It may also be important to design a UI which can support dozens of jobs running in parallel. For example, our server nodes scale up and we can support many files at a time. Users will need an effective way to view the status and manage them. In 2011, we revamped our UI and this is what we came up with.

---

Here's a 30-second video showing how the whole process works, and how the interface responds to the current status of the job(s). If you want, you can try for yourself starting at this page (free, don't worry...).

---

Screenshots of the workflow... takes about 30 seconds to a few minutes, depending on your files.

Upload your list

See the file uploading

Map the fields (happens automatically; just confirm that the auto-map was correct)

Stay current with what's happening now

---

Notice that as the list begins processing, the user can go back and upload more, which lists will be handled in parallel, in real-time. Lists can be cancelled/deleted at any time, and when finished, they can be downloaded. A checkmark appears when the list has been downloaded at least once.

This is actually an aging design, which we're making plans to upgrade. It'll feel much more like a modern, responsive web app.

It's worth another note that we've changed the look of the account "dashboard" page many times and this design has initiated the fewest support calls/emails, and customers like having it all visible at-a-glance. First-time users tend to find they become comfortable with it pretty quickly.

Answer 2

I'm unaware of a concrete example, but I'd consider the following:

It sounds like you want a main page listing jobs that have been entered into the system. Each job should have a status (e.g. pending, running, completed) which is clearly labelled, since that's probably the most important piece of info. You might even want to clearly separate jobs by status - for example, display them in three separate lists.

Each running job should show its current duration and estimated time of completion, if possible. Completed jobs can show duration taken, and pending jobs could show estimated start time, if possible.

If jobs are per-user, you'll want to consider whether one user can see another user's jobs, or not.

You'll probably want 'job control' actions (e.g. stop a running job, delete a completed job, etc.) You might also want some kind of priority system to control the order in which jobs will be processed. Or simply display them in a queue and allow them to be ordered with, for example, drag and drop.