I think a striped rugby shirt would look best with a solid colored kilt.

You did match the colors well

I think the hose up/shoes look fits you better than wearing hose down/boots just because it's "casual"...