The limits are violence, no? The point of free speech is that the government doesn’t get to use violence against you for words. When that turns into actions such as violence or credible threats, then self defence and law enforcement come into play.

If you meant that their intolerant language should warrant intolerance from you, then great, that’s covered under free speech. But that’s not what you mean, you want to respond to intolerant language with violence and want to pick which language is intolerant enough.