"Stitch Witchery"
This is not an interfacing. Stitch Witchery is what's called fusible web and used as you described.

regardless of Dixiecat's points (because in time the fusible does leave a dimpled surface after time and sewn in doesn't)
I can only assume that you've used low-quality fusibles and/or applied them incorrectly and/or used the wrong fusible for a project. I have made quite a few garments using these products and have had no problem with them when used properly. I'm totally serious.

And yes, the answer really is, don't use fusibles in kiltmaking!! The best product is the hair canvas.