I think we should keep the feature too, I'd hate to ignore someone because I didn't know they replied to me.
Besides you are not FORCED to click on them. Only downside is that your notifications would always have a number next to it. Its not really a big deal..