There is no situation where I would hear the word "skirt" and think of a garment intended for a man.
I could. Sarong. Lava lava. Sulu. Fustanella. Kikoi. All are skirts. All are worn by men. Many are worn by very manly men from warrior cultures, all of whom could probably rip you a new one for calling them skirt wearing sissies.

My two pesos.