Are there any UX reasons for ChatGPT staggering replies word by word?

Question

I am not a machine learning or artificial intelligence specialist by any stretch. I don't know how the back-end or programming does any of its stuff, and how it generates responses to send back to users.

I do see, however, that ChatGPT sends back its responses with staggers and delays. There is a thick cursor that makes it look as if someone is typing on a low resolution monitor. And the responses are given back to us word-by-word, not all in its entirety at once.

Now my impression: this feels like there is a conversational design reason behind it. It might be to humanize the bot. Humans don't type and speak in all entirety at once. We have to type, word-by-word, or move our tongue.

Does anyone know why ChatGPT does that stagger? Can anyone with probably an artificial intelligence background also chime in as to why the AI staggers its responses? Is it purely for mimicking conversation? Or is there a back-end reason behind it?

Answer 1

The "why" generally is unanswerable. It may be that they did extensive user testing and found that chatGPT feels more friendly this way, it may be that it's a good way to rate-limit people, it may be that Dave from engineering wanted to show off.

We can determine however what effect it has on people (or in other words, what the user is experiencing). And indeed, it feels more like we're up against a typer - reasonably fast, but still within the realm of human capabilities, who uses human language and phrases to converse or, well, chat with us.

Before ChatGPT, support agent or marketing chatbots would use chat bubbles and "typing indicators" for each sentence to stagger their responses. Here's a random example I found on the web:

For ChatGPT, since it tends to throw multiple paragraphs at once at you, it might invoke more of a "20 missed calls from mom" feeling if this idea was used. ChatGPT's "typing", while not our daily experience of others in chat, is very similar to our own typing.

So, in other words:

• Did OpenAI intentionally make ChatGPT good at imitating human typing by staggering the message? We don't know.
• If you want to make a conversational AI yourself, should you copy ChatGPT's behavior to make it more human? Probably yes.

Answer 2

Are there any UX reasons for ChatGPT staggering replies word by word? [...] Is there a back-end reason behind it?

Because ChatGPT is autoregressive (=generates each new word by looking at previous words). On the OpenAI Playground you'll see that words are displayed faster when using smaller models such as text-curie-001.

ArtificialAva on Twitter compared the display speed of ChatGPT vs. ChatGPT Plus vs. ChatGPT Turbo Mode and showed that ChatGPT Turbo Mode is over twice faster to display the output, which further indicates that ChatGPT shows its response word by word due to its backend (computation time + autoregressive).

Note that Google Gemini displays the entire output text at once, and that ChatGPT Plus advertises "Faster response times". Also, the OpenAI ChatGPT application for Android outputs text much faster.

---

Update (2024-07-25): gpt4o-mini has a much faster output:

as well as on llama 3.1 8B-instant on groq:

Answer 3

Here is what ChatGPT says to you)))

You are correct that the way ChatGPT delivers its responses with staggered delays and a typing cursor is intentional and serves a conversational design purpose. This approach is known as "conversational UI" and is intended to mimic the experience of having a conversation with a human being.

There are a few reasons why this approach is used. One is that it can help to make the interaction feel more natural and engaging, as it creates the impression of a back-and-forth conversation with a human. Another reason is that it can help to manage the user's expectations and keep them engaged by giving them time to read and process each response before the next one arrives.

From a technical perspective, the delays between responses are often added using various techniques like random delays, model sampling time, and other optimization methods, in order to give the impression of a more human-like conversation flow. However, the specific implementation details can vary depending on the platform and the specific use case.

In summary, the use of staggered delays and a typing cursor is a deliberate design choice in order to create a more natural and engaging conversation experience, and is not necessarily driven by technical considerations alone.

Answer 4

Staggering does feel more natural, if you try the new Bard AI, it pauses then instantly produces the full answer which feels odd after ChatGPT.

ChatGPT produces a steady strem of text and a fast reading pace hence it feels more normal.