When to collapse
If under the content, which contains all the comments, follows other content. So the user can either read the comments if he's interested or scroll to the next content.
Example Facebook:
Facebook collapses the comments on the timeline. This is because the user usually scrolls through all the posts and only read the comments of those posts where he's interested in.
When to show it all
If there is no relevant content that follows the comments. This is mostly the case when you are in a detail view of a post or article.
Example News Site:
In an article on a news site, at the bottom is mostly the comment section where all the comments are shown. This is because after an article, mostly you go back to the overview, unless you're interested in the comments. So they display all comments expanded, in order to let the user scan through the comments and maybe step in to the conversation.
Don't forget the quantity
Maybe you think "but there are websites which collapse comments on the detail page", like quora.
Here you need to think about what kind of comment the user will post. Do the comments itself represent something worth knowing? If yes, the comments on this comment are a conversation about the answer and not primarily interesting for the user. So collapse it and set the focus on the important comment.
On reddit or also Medium, the answers aren't collapsed, because those are more of a conversation between users about the topic of the content. So it could be more interesting for the user to read all comments without the extra click to expand.
mobile / desktop
A more obvious point is the mobile / desktop talk:
You have more space on the desktop to scroll through a collapsed conversation. On mobile, the site could get extremely long to scroll through, so maybe collapse the comments there.
Final though
Think about what you want to achieve. Then look up similar applications to yours and see how they've done it. And think about why they done it that way and what the users want to achieve.
There is, as always, no "one fits all" solution.