I believe that accessories are about 90% of the formal look. Obviously there are some kilts that cannot be "dressed up" to formal. Most, but not all, kilts will look great paired with a prince charlie and dress sporran. A handsewn, all-wool kilt even more so, but a more casual kilt will often pass muster.

Part of the issue not previously mentioned is the AUDIENCE present. If you were to go to a Burns Night with several knowlegdeable (and well dressed) kilties, a casual kilt would stand out more. If you were to go to an event where you were the only one kilted, then a casual kilt with formal accessories would be fine. I believe that 95% of people would never know the difference. Would it matter to you?