Yea, I agree. The Prince Charlie is the kilt equivalent of tuxedo. Black Tie. Always worn with tux shirt and black bow tie. So if the invitation does not specify tux for the non kilted perhaps you could ask the organizer for clarification.

But do what the invitation specifies and change into a rental Prince Charlie or dress in a tux for the dinner.