The self consciousness comes and goes.
I can relate to this. I don't wear kilts all the time, but I've been wearing them for over 35 years, and I still have times where I feel self conscious wearing one.