You could swing it to the side... but I've also undone the buckle and taken it completely off.

.