I wear an aloha shirt and pith helmet with mine. Since I get away with that (though that might have something to do with my inherent eccentricity [like writing kilted parodies of Gilbert and Sullivan]), I think you can get away with a t-shirt.