Sorry, I just couldn't resist... I'm not sure I would have been as circumspect as you were. I probably would have drawn on the guy. Good argument that it would have been justified to do so. I don't know.
Before CHL and MPA I had a couple incidents of people "attacking" my car while I was in it. I came very close to "mule kicking" my door open as hard as I possibly could and knocking the attacker on his rear.