Another factor to consider is the place you live. I remember when I first came to Oregon from NYC; I was thin, wearing high waisted trousers, suspenders, and nice pale complexion, and for the life of me could not figure out why people were staring at me. Now people stare at me a little more respectfully. :P

I like the tweed jackets or a nice waistcoat quite well, but probably wouldn't wear them where I live unless I was going to a wedding or something similar.