You will know best about your family so I can't comment about that.
However, I have worn the kilt regularly in Devon, Somerset, Sussex, Kent, Gloucestershire, Worcestershire, Warwickshire, Northamptonsire, Cambridge, London (especially the last two), Hertfordshire, (not Hampshire or Dorset for some reason), without a moment's problem. That's leaving aside Wales - Pembrokeshire, Carmarthenshire, Powys, Glamorgan, Monmcuthshire.
As Ann says, there are some parts that sensible people should avoid - the same is true all over the world - in whatever circumstances one is. For me, country pubs are one thing, certain urban taverns another; I have never been to a night club and certainly never will. Anywhere where people are likely to get drunk is a place to avoid as far as I am concerned - regardless but especially in a kilt. And in England and Wales the centre of towns and cities on a weekend at night are often populated by largely inebriated punters.