Really depends on the application, either could work as long as the user was informed.
A local newspaper I read allows users to comment on stories.
The comment section is heavily moderated, and any comments that do not meet community guidelines are removed.
Comments that a removed are immediately deleted. The OP is notified of when/ why, and that is the end of it.
The user is now aware that the comment has been removed, and there is no further action available/required on their part. Hence, there is no reason to keep the post/comment around.
This also helps keeps a clean and simple site.
An outdoor trail website I use allows users to upload trails.
Trails contain a lot of information, everything from photos, directions, gps co-ordinates to maitenance/usage logs. It can take a significant amount of time to create a post/trail.
These trails are moderated by admin.
Often, if trails that are uploaded and don't meet requirements(duplicates,too short,not actually a trail, etc), moderators will remove them.
The trails/post are removed publicly, but from the OP's perspective, the trails/posts still exist and are clearly marked disabled/removed with the moderators comments.
This gives the user a chance to understand why there post was removed, and gives them an opportunity to modify it to meet requirements and re-post.
In this scenario, there is a reason to keep the posts, as further a actions can be taken. Furthermore, because of the amount of work that goes into creating a post in this scenario, it could be really annoying for users to have to re-create and re-post because of what could be fixed with a simple modification.
Depending on the scenario, either could work. But as a rule of thumb, if there are actions a user can take with these posts after a moderator removes them, then there is a clear benefit and reason to hold onto them. If there is nothing a user can do about the removed/delete post, then there is no benefit to keep them. But in both scenarios, the user needs to be clearly informed of when and why this action took place.