I doubt 99% of folks would recognize a beret as being specifically military, unless it had a unit badge on it. It would be no more military than wearing surplus cammies devoid of all military insignia. Now the traditional US Navy bell-bottomed uniform parts would be more difficult to pass off as strictly civilian.