In the Guards we used to wash ours fairly regularly. Seemed to do the trick, certainly didnt do any harm.

However, two factors there

1. Our sporrans took a fair bit of bashing between beer and food

2. General motto was "if it's wet, it's clean"