I use a variety of kilt pins, some antique, some bought, some I made, and I have had several that would fall off at every chance. I've found that pierced ear-ring retainers, little clear rubbery-plastic knobs that you can get at most ear-ring stores, about twenty to fifty cents each. I put the pin through the kilt once, slide the keeper on and up nearly to the top of the pin, put the pin back through the cloth to the front of the kilt and latch it, then slide the keeper up to the top of the pin. It may still come unlatched, but it will NOT come off the kilt, as the kilt pins are thicker than ear-ring pins, and thus fit much more tightly.
Try it.