This is a close cousin of “if the shoe fits, wear it”. It’s a phrase related to people who cannot take personal criticism without taking it personally. All it usually takes is an off comment by a colleague at work, or a snide comment in a social setting. Boom and this person goes off. Sometimes directly at the person who made the comment and most of the time these folks use it to gossip. Here is something to consider:
If you complain and whine about it then there may be something to the comment and you need not complain about it because you just validated the point of the statement to be true or at least possible.
Avoiding this uneasy commentary can be difficult mostly because of one reason. This can happen when you start taking things too literally and seriously. In other words your ego is high-jacking your rational thought. Do not let this happen to you. Turn this saying around: If the broom does not fit, well, then don’t ride it! It is perfectly ok not to dignify criticism with a response. Give yourself some time to think about it and how you want to react to it.