I think I would just turn slowly and with no facial expression at all, look them in the eye and wait - a long time if necessary. When somebody does that, the ball is in their court.