Kid, amen to your post, I was raised the same, + I like hats.
But you know why people don't remove their hats indoors? There's no where to put them! If you see old movies from the 30s and 40s where everyone wore a hat, if a man goes into a club or resturant, there was a hat and coat check. Even homes would have a coat /hat rack in the entry hall. Nowadays if one goes in a nice resturant wearing a hat and coat, you are on your own to find them a home at your table, along with your umbrella if it's rainy!

To finish up, a quote from the Grand Master;

"Moving parts in rubbing contact require lubrication to avoid excessive wear. Honorifics and formal politeness provide lubrication where people rub together. Often the very young, the untraveled, the naïve, the unsophisticated deplore these formalities as "empty," "meaningless," or "dishonest," and scorn to use them. No matter how "pure" their motives, they thereby throw sand into machinery that does not work too well at best."

Robert A. Heinlein