In my ideal world, you would let the user choose between no notification badges, a simple "on/off" badge1, or a numbered badge.
As Ainhoa Ostolaza's answer notes, numbers can "add urgency" to notifications. For some apps, for some people, that's good. For other apps, or for different people, that heightened sense of urgency is bad. As with many choices, it is very unlikely that there is a "one size fits all" answer, hence my preference for letting the user decide.
For example, the email app on my Android does show numbers, and I prefer that. In my case, the majority of emails come from a mailing list so unless I'm expecting an important email, if I'm "doing other things" (including doing something that's nothing to do with the phone), I will often ignore the notification until several emails have arrived and deal with them in one go (mostly just read-and-delete). Having a binary "you have an unspecified number of emails" badge wouldn't be as helpful.
On the other hand, if I was getting mostly work emails, that needed to be responded to "as soon as possible", then I wouldn't really need to see a count (the assumption being any email should trigger a response). In fact a count may be detrimental since if there was a reason I couldn't respond immediately, the count would only serve to increase stress from not being able to respond.
Similarly, there may be other apps where I really don't care whether "something new" has happened or not, so I would want to turn of badges altogether.
You mention "just using a red badge
". There may be cases where graduated (but not numbered) badges could be useful: green for "something's new (but there's no urgency)" through to red for "deadline approaching" (subject to the usual caveats
about relying on colour alone).