I don't think it's the size of the man as much as the size of the kilt. I wear a kilt belt from SWK (I think it's 3" but I could be off) with all my kilts, but I have worn a typical 1.5" or whatever belt with my sportkilt. It only looks odd because it's so much smaller than the belt loops. If I were to try that with my other kilts which are all 24" or so, it would just look wrong.

Earlier this year I got a friend to wear one of my kilts for a few hours, but since I only had one kilt belt, and it needs a few days for the leather to accept an adjustment, he had to wear a regular belt. A smaller kilt might have been able to pull it off, particularly if it had the appropriate belt loops, but in this case, well you be the judge: