I get bugged by those comments but most of the time it comes from women who honestly don't know better. Men almost always call it a kilt when they see me in it. To the women who misspeak I typically respond kindly and with patience as they speak from ignorance rather than malice and they are almost always complimentary rather than mean. I usually tell them it is a kilt and thank you.

If being teased by my less secure male friends and they say "nice skirt" I say "thanks, I like it".