I am working on a CRUD application for describing tumor models. A tumor model is the description of an experiment an oncology researcher did with lab animals. There are a lot of possibilities of what this experiment can look like, so no experiment needs all the possible data fields. Therefore, we have many optional fields.
We tried finding common types of experiments which share the same fields and creating separate forms, but found out that this will not work, because practically any combination of fields is possible. We also tried to push for less fields, but the stakeholders are against it, and vetoed it over our protests.
So now we have a situation where the user sees many fields, including optional ones. They are highly interesting sometimes, but will not be needed most of the time. I had a user test today, and she was just trying to come up with something to enter into every field she saw. She was aware of the fact that some fields are marked as required and the rest are not, but she still hated leaving optional fields empty. One field in the description of the experimental animals was "phenotype", meant for entering interesting deviations from normal (e.g. congenitally blind, or resistant to some carcinogens). She went to the site of a breeder selling experimental mice and copied the backstory of how the mouse was created, the whole page of it complete with formatting, into that 3-line field.
But we also had some "hidden" fields. We have a tumor description, and there is no field "metastasis" visible, there is a button "Add metastasis", which adds a rarely used field to the page. She had to use it, and instantly complained that it is not visible and she has this additional click. So this is a solution which seems to annoy the users.
It is better for both the user (who already has to enter a lot), and for the database owner (who doesn't care for the backstory of a mouseline) if the optional fields don't get filled for models where there is nothing interesting to say about them. But how do I get the users to discard the mindset "This is an empty field, I have to fill it" without hiding it behind a barrier?