It's not my lady I would be worried about, it's the 'ol fella'
I slide to the side also, having the sporran on a leather strap makes this easier. Also, make sure there is nothing heavy in it.