Avoiding offensive language in publishing App

Question

I am working on a mobile app that allows the users create long videos picking up small clips.

Then the user selects a title for the video and it is published in different networks (Facebook, Youtube, etc. depending on the user's accounts).

The problem is that my client wants to avoid rude language in those titles. I suggested filtering words using a dictionary, but it looks it is not enough. The meanings of some titles, even without rude words, may be offensive.

To solve this, they want something like a column with articles and prepositions (A, an, the...), then another with nouns related with the clips, another with verbs...

So the user picks up the words and compound a title: The + car + is + blue (silly example, sorry).

ARGH! Any ideas about how to solve the problem without using this interface?

Thanks!

Answer 1

The seeming simple task of not allowing rude words and phrases is actually incredibly complex to solve with just UI.

Donald Norman stated that if an error is possible, someone will make it. And I would suggest extending that rule to include if a rude word or phrase is possible to be said it will be.

The problem is with our language. We have concepts like sarcasm and innuendo in which context or intonation can give very different meaning to words or phrases. Even capitalisation within the same phase can have a big effect.

Take these two phrases for example:

Helping your Uncle Jack off a horse.

 

Helping your uncle jack off a horse.

Two letters are capitalised differently and a whole world of difference in meaning.

And lets not forget human ingenuity, especially in the young. People are incredible at bending a system not designed for a particular purpose to their will. This is where the whole language of ASCII art came from as did emojis.

Think your safe in church? Think again. Many years ago I worked on a project called Church of Fools. It was a 3D environment where people could interact, worship and pray. Two of the actions you could make your avatar do were kneel on the floor and pray and another where you would stand and throw your arms in the air hallelujah style. Perfectly reasonable safe incorruptible actions you would think. Until that is, some bright spark realised that if you had someone kneeling in a praying position in front of someone throwing their hands in the air hallelujah style it looked more like a porn movie than something that would happen in a church. In that instance even words weren't necessary.

There have been notable successes however they come at a price. Disney have a social network for young people called Club Penguin where communication is restricted to a few words and some symbols, that level of restriction may be appropriate for kids but it's unlikely to be workable in your case.

So back to your situation, the more scope you give people, the more likely they are to do something you don't want them to. Trying to create a UI to solve this problem is probably a losing battle.

The vast majority of successful implementations of rudeness checking are done manually through moderation. Usually post-moderation with a report function is the most cost effective. Though which you implement depends on your audience.

Answer 2

Here's a radical concept: Human filtering.

Tell the client. "You want granular and subjective control of content? Humans do that best!"

The very nature of subjectivity is that it's subjective. An authoritarian attitude towards content's nature, terminology, phrasing, meaning, interpretations, influences, origins, inferences and every other nuance is impossible to program for, or to.

And even if it were, the process of monitoring its settings effectiveness (and lack thereof) in all aspects of subjectivity would be more involved (time and energy) than humans doing the filtering under a hierarchy of referral for any and all contentious issues.

The best part about employing humans to do filtering, recommendations and editing is that you're employing people, paying them, developing them and gaining a team of people increasingly attuned to and sensitive to your concerns and motivations.

Doesn't matter how well you program a computer to do a task for you, it's never going to pre-emptively be considerate of other factors and interests pertinent to your life, business and communications. Your team, on the other hand, will become increasingly loyal, effective, considerate and caring.

Caveat: Requires empathy to deal with others in a manner truly beneficial over the longer term. Those interested in databases and control of communication usually don't have much confidence in others. The origin of that distrust is often a lack of empathy.