3

I am not a machine learning or artificial intelligence specialist by any stretch. I don't know how the back-end or programming does any of its stuff, and how it generates responses to send back to users.

I do see, however, that ChatGPT sends back its responses with staggers and delays. There is a thick cursor that makes it look as if someone is typing on a low resolution monitor. And the responses are given back to us word-by-word, not all in its entirety at once.

Now my impression: this feels like there is a conversational design reason behind it. It might be to humanize the bot. Humans don't type and speak in all entirety at once. We have to type, word-by-word, or move our tongue.

Does anyone know why ChatGPT does that stagger? Can anyone with probably an artificial intelligence background also chime in as to why the AI staggers its responses? Is it purely for mimicking conversation? Or is there a back-end reason behind it?

1
  • ChatGPT does this because this is how ChatGPT actually works. It predicts one "token" at a time, based on your input and its own output so far. Usually a token is a several letters long. If it's always doing it word by word, it could be that they made it wait for a whole word to be generated so it would look a little less strange. The alternative is to wait for the entire answer to be generated, but that takes too long. Commented Jul 16, 2023 at 20:46

4 Answers 4

5

The "why" generally is unanswerable. It may be that they did extensive user testing and found that chatGPT feels more friendly this way, it may be that it's a good way to rate-limit people, it may be that Dave from engineering wanted to show off.

We can determine however what effect it has on people (or in other words, what the user is experiencing). And indeed, it feels more like we're up against a typer - reasonably fast, but still within the realm of human capabilities, who uses human language and phrases to converse or, well, chat with us.

Before ChatGPT, support agent or marketing chatbots would use chat bubbles and "typing indicators" for each sentence to stagger their responses. Here's a random example I found on the web:

Landbot

For ChatGPT, since it tends to throw multiple paragraphs at once at you, it might invoke more of a "20 missed calls from mom" feeling if this idea was used. ChatGPT's "typing", while not our daily experience of others in chat, is very similar to our own typing.

So, in other words:

  • Did OpenAI intentionally make ChatGPT good at imitating human typing by staggering the message? We don't know.
  • If you want to make a conversational AI yourself, should you copy ChatGPT's behavior to make it more human? Probably yes.
4

Are there any UX reasons for ChatGPT staggering replies word by word? [...] Is there a back-end reason behind it?

Because ChatGPT is autoregressive (=generates each new word by looking at previous words). On https://beta.openai.com/playground you'll see that words are displayed faster when using smaller models such as text-curie-001.

https://twitter.com/ArtificialAva/status/1624411499375603715 compared the display speed of ChatGPT vs. ChatGPT Plus vs. ChatGPT Turbo Mode and showed that ChatGPT Turbo Mode is over twice faster to display the output, which further indicates that ChatGPT shows its response word by word due to its backend (computation time + autoregressive).

Note that https://bard.google.com/ displays the entire output text at once, and that https://openai.com/blog/chatgpt-plus advertises "Faster response times". Also, the OpenAI ChatGPT application for Android outputs text much faster.

2
  • How it generates text in the backend may not be how it presents text in the front-end. Commented Feb 24, 2023 at 4:26
  • That's correct. Commented Feb 24, 2023 at 4:29
0

Here is what ChatGPT says to you)))

You are correct that the way ChatGPT delivers its responses with staggered delays and a typing cursor is intentional and serves a conversational design purpose. This approach is known as "conversational UI" and is intended to mimic the experience of having a conversation with a human being.

There are a few reasons why this approach is used. One is that it can help to make the interaction feel more natural and engaging, as it creates the impression of a back-and-forth conversation with a human. Another reason is that it can help to manage the user's expectations and keep them engaged by giving them time to read and process each response before the next one arrives.

From a technical perspective, the delays between responses are often added using various techniques like random delays, model sampling time, and other optimization methods, in order to give the impression of a more human-like conversation flow. However, the specific implementation details can vary depending on the platform and the specific use case.

In summary, the use of staggered delays and a typing cursor is a deliberate design choice in order to create a more natural and engaging conversation experience, and is not necessarily driven by technical considerations alone.

3
  • ChatGPT has been known to hallucinate and present incorrect information with absolute confidence — it's an LLM. This might not be the truth even though it comes out of itself. Commented Feb 27, 2023 at 2:06
  • I guess if you want to know about technical part of ChatGPT better than ChatGPT you will not find.
    – AndriiD
    Commented Feb 27, 2023 at 12:00
  • 1
    "You are correct that the way ChatGPT delivers its responses with staggered delays and a typing cursor is intentional and serves a conversational design purpose. " Source? Commented Mar 7, 2023 at 4:42
0

Staggering does feel more natural, if you try the new Bard AI, it pauses then instantly produces the full answer which feels odd after ChatGPT.

ChatGPT produces a steady strem of text and a fast reading pace hence it feels more normal.

1
  • Welcome to UX StackExchange Adrian. Can you expand on the answer a little more to reflect the UX decision/ your speculation about it? Commented Jul 17, 2023 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.