Merging huge data

Question

I've searched UX.SE and other resources but not able to find any solid answers on how to merge N multiple records.

Here is the attached screenshot of application which needs to be revamped.

Before you see the screen, pls consider below things:

1. Who will be using application: User will be Admin, whose job is merge duplicates or non duplicates data quickly and efficiently

2. Limitatation : There is no limitation on number of merging records. There could be N number of duplicates at a time.

3. Each records could have 50+ fields.

In just previous screen user is selecting 8 records which he wants to merge and move to next screen, 2 more columns are being added in above screen which creates mess with horizontal scroll. Also there are long fields value like "Gentsesteenweg" or long email ids ([email protected]) which increase col width.

I would like to get rid of this and want to create best user experience to help user to quickly see, edit, merge and leave the screen.

I have gone through Gmail, yahoo and other applications and performed duplicate merge, but they don't fit the bill. Even followed : Design pattern for merging duplicate contacts?. Merge multiple table entries And picked some suggestions but as my problem is different, hence need different solutions

To reduce the clutter in preview screen, purposing below designs with number of duplicates before the merge, so that user can select which records he want to merge.

But there are again a problems like if user wants to see preview (by hitting small down arrow) it will create huge Vscroll. I Assume that should be acceptable.

In merged preview screen, I'm only showing merged records. But the problem is now user can not select any value form either of records. He must have to edit info which is again take good amount of time.

Existing design:

Purposed design:

Imporved design

In below image, merged data is shown on the left along with conflicts on the right. I'm curious to know about the alignment of conflicts tables vertically or horizontally. Any thoughts?

Answer 1

Actually, if your goal is to merge fields and avoid duplicates, I think that you could think of this as two separate tasks:

• Spot Differences: An action for the user to do, which can only include the different fields. Unless there is an actual reason to show all data, why don't you show only where the entities differ? For example, if the email is the same in all duplicates, why even bother to show the column at all? As a user, I would like that to be automatically merged. It should also remove visual noise from the UI: less columns! Concerning examples of UIs showing differences, some can be seen if you search for diff merge.

Assumption: The user has had a look at the data before and knows the fields well; i.e., if the email is missing they'll know it means it's the same in all records.

• Merge records: The technology should take care of that, you can present just the merged entity and the user should only be able to choose [Yes, go on | No, go back | Maybe edit the merged record (if that makes sense in your app)?]. It's not the user who should do the actual merging. The user should be able to say if they want to keep or discard something that's different; merge everything else, please.

As a conclusion, I don't think you have to show everything. Show only what the user wants to know; if 10 entries of 75 fields each differ in just two fields, that's the ones the user should care about.

Answer 2

Your purposed design (screen 1 - 3) is already a good starting point. Screen one might remain unchanged and here I would start to simplify the merging process for the user.

Preliminary considerations: There are two different cases.

Case 1: Data can be merged without conflicts, so merging is not dangerous and no data can be lost. Actually, it is not neccessary to involve the user in the merging process. Rather, the user should be relieved from the effort to controll and monitor the merging process.

Case 2: Due to overlapping values there are conflicts requiring the user to decide which values are to be kept in the merged record. In this case, the user must be incorporated, but he should only be confronted with overlapping values and not with the whole record.

Conclusions: I suggest to add an action button "merge" in each row in screen 1:

^{download bmml source – Wireframes created with Balsamiq Mockups}

The color of the button should indicate, whether the merge can be carried out with a single push of a button (green) or whether conflict resolution will be necessary (yellow). A click on a green button should immediately trigger the merging. A click on a yellow button should bring up a dialog for conflict resolution (maybe you can use some hints from the SE answers you referenced). This way the user can quickly click through the green buttons. Afterwards, he can concentrate on the remaining conflicts. This only works provided the computer determines in advance for each group of merging candidates, whether case 1 or case 2 is true.

Further suggestions: The user should be able to undo the merging actions (at least until the changes are saved).

Answer 3

My answer is too in lines with Anna. Only difference is that instead of directly merging, I am assuming user will view the records and merge.

So after the user has been shown how many duplicate hits there are, show the break up by record. Given the user and option to view the records and merge the ones required.

Optionally, there might be records that the user might want to 'un-tag' or 'mark as invalid for this user'

Answer 4

In reply to

Lets say he selected 10 records which he thinks are duplicates and hit the merge button. Now question is how to show 10 records on next page.

I still think the UI can be made more simple. Visually showing the user what's being merged into what is always a good idea. While your tabular method is a good start, simplification is always better.

Expand a pop-over upon click to show the details, let the pop-overs be used to compare the information during the merge operation. Use the delete button to remove a record from the merging process. Show what the merged records look like. Allow user to edit if need be.

Here's an idea to sleep on. use an appropriate merge icon instead of my poor shuffle substitute.

^{download bmml source – Wireframes created with Balsamiq Mockups}