"I think that many would secretly like to wear it more often ..."

I am sure that is true.
The difficult part is finding an excuse to wear one.
Thus they take up the bagpipes or country dancing etc.
I have always been surprised, for example, to see how the men all get kilts when they take up country dancing, even "foreigners" with no connexion with Scotland, and even if they would not dare wear it for any other activity.
There is a gent here who, I'm sure, took up dancing just to have an excuse to buy a kilt; once he had it, he gave up dancing!

Martin