A lot of bad advice

One you should have never caved in but told the manager to go seek counsel with his superiors as you have been expressly allowed to were what you did. Furthermore you should have reported the tirate to the manager and demanded an apology both written and public from the offender. failure to do so would require you to report this incident to the regional manager, the companies HR department and the state labor board. This is precisely what there are harassment policy and laws you were harassed and use the term a lot in all of your discussions.