too nice?

So do I have a sign on my forehead that says "I'm nice so be mean and nasty to me." ?
Twice today I have had customers swear at me because I wouldn't do something for them (for various reasons). What is with people?
My family sometimes tells me I'm too nice. But I don't want to be rude.
I like people. I like to be nice to people. I like to be patient with people. I can be firm when I have to be and I won't take someone pushing me around.
But is my niceness and patience to patient and nice? When is being nice not enough or more than enough?