Andrew, for a very brief (several hours) period I owned a Victory Kilt in green.

My wife felt the same way about it, she used the word "skirty".

Frankly, while I think you look great in your kilt and others who wear plain kilts mostly look very fine in them, for me - in Australia it doesn't work.

Have you considered a tartan kilt? If you don't have any clan links there are tartans you can wear, Black Watch, Stewart Hunting, Australian tartan and others.

My feeling is that in Australia a tartan kilt sends a clear and undeniable message that this is a male garment.
However, because we are somewhat blinkered in our perceptions, the plain kilt may not send that clear message.

Depending on how you feel about tartan, this may be another way around your wife.