1

I'm working on an application that has a few CSV imports. I'm trying to figure out if the import should be all-or-nothing or if the import should accept all of the valid records in the CSV and reject the invalid ones.

It's possible a record is invalid because it violates a unique constraint.

The benefit of partial imports is that you get your valid data immediately while you try to fix your invalid records.

Some downsides are that you'll need to modify the CSV to remove the imported records. If you had a unique constraint violation within the file, then the first one in won though it may have been the incorrect record.

What seems to be the best (or at least most common) way to handle validation errors in bulk imports?

4

We recently handled this issue by making it as easy as possible for the user; import what is importable and then provide a CSV of the rejected files only so changes can be easily made for a second import. We actually notify the user on initiating import that some records will not be imported (and provide the CSV for download) and at the end we also give them another opportunity to download the CSV.

enter image description here

2
  • Oh, that's a good idea. I like that a lot.
    – D. Patrick
    Aug 4 '16 at 23:51
  • Nice design though you need to look at the contrast of the text, hardly readable
    – Mervin
    Aug 5 '16 at 21:15
1

It really depends on the data being imported. Data where records relate to each other or impact each other should be all or none. I'm speaking from experience of what tends to work best in the financial industry.

All or none is generally easier to reason about. Fix the broken records and retry the file.

Some means that you now have multiple batches - just re-upload these broken records. For users that want to maintain a copy of what they uploaded or re-use an upload, they now have a version that partially uploaded and another file with records that didn't make it the first time.

A clearer mental model is to:

  • Indicate that the file failed due to errors with (X) records.
  • No records were imported
  • Make available a download of the file with an indication of which records caused the upload to fail.

Users can then choose to split out the offending records and fix them later or fix the records in the file and resend it. The file they upload that works is a good archive and starting point if they need to reference what they uploaded or do a similar thing again in the future.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.