Our sense of fashion

[Cygnus]

North America (and the USA in particular) tends to be a place where relaxation of standards seems to be the norm, and as those standards are relaxed, somebody decides to relax them even more in a never-ending cycle that can only result in people wandering about in the buff (okay, so that's a bit of an exaggeration, but I hope you can see what I'm saying). In the not-too-distant past, and in the present in some locales, a collared shirt, tie, and tweed suit are what is worn by outdoorsmen when shooting, fishing, etc. It's actually a look that I much prefer to the ubiquitous camouflage and bright orange so common around here. It's also quite likely that your father/grandfather wore a suit and tie nearly everywhere he went, so it's not as if the age of manners and "gentlemanly" dress are that far behind us.
In my opinion, there is nothing wrong with doing something because you've been taught that's how it's done - that is tradition. It doesn't mean you should blindly follow everything you read here or elsewhere, but it doesn't hurt to be aware of what is considered proper and traditional before modifying it to suit your own taste.
Taking it back to kilts, most of what constitutes traditional highland attire comes from the recent era of suits and ties, so "casual" dress still includes a tie and jacket, as does traditional "hunting" dress. There are many on the forum that prefer this style, despite the fact that many of our T-shirt-clad friends and fellow X-Markers consider it a bit antiquated. If that isn't your style, by all means, wear a pullover, T-Shirt, combat boots, etc. with your kilt. It is yours, after all.