I'd have to see the white blazer to be convinced it didn't look "good humorish"
I would just go with a white shirt and crimson tie to get your school colors.
I don't think pants and jacket have to (or should be) white or crimson.