From experience and from what I have seen.. words really can hurt people. Sometimes more than physical harm. It's a real psychological problem..
Yes, they do hurt your feelings, and when carried to far, they can attack your integrity. And there has to be a point where you can have this stopped if you can't stop it yourself.
The saying meant that if you are attacked physically, then yes, you need to defend yourself physically. If the idiot is only calling you names, then violence isn't the answer.
But, you are right, no doubt.