Oh, certainly, if someone were to wear a specifically military pattern rosette, it might be taken as less-than-complimentary by persons who put in a lot of effort to earn the right to wear it. I was thinking of something along the lines of a rosette that wasn't an exact copy of a military one.

But yes, certain things like wearing a red hackle behind the cap badge on a Glengarry or Balmoral (something which the RHR would take a dim view of) might get one a special "discussion" in a back alley, similar to how wearing a maroon t-shirt in the proximity of certain heavily armed folks who jump out of planes on the way to work might be detrimental to one's dental work.

It might be worth researching whether the rosette one is considering wearing is in the pattern of any military ones first.