Geoff there are I am sure people who's beliefs vary. I was raised with a more old world belief where there were certain rules of etiquette that should be abided by a gentleman. These rules were hard and fast, one did not swear or use coarse language in the presence of a lady, one always held the door open and one always removed ones hat as both a consideration of others and as a sign of respect. While not all of the rules,these are certainly some of the more commonly applied if you will. I respectfully see no need to wear a balmoral or any other hat for that matter while sitting down to dinner in a restaurant. Perhaps I have missed something?
I can recall that even in the military you were obliged to remove your headdress when entering the mess in all but a few cases, again religious requirements exempted. I personally believe that the world might be a better place if there were perhaps a bit more attention paid to courtesy and etiquette. That however is a discussion, the finer points of which are best suited to take place over the aromas of a good single malt!