Indeed. There's never a wrong time to try to be polite.being polite never hurt anyone
being rude is easy
learn to behave is hard
Sir/Ma'am sounds way too submissive in this day and age for adults. The only exception is military or government.
I say sir to strangers here and there for fun and being polite. That's about it.