I think that both intent and perception are important, and they are both judged in the more modern understanding of "toxicity". Malintent is a direct "strike". But, lack of malintent doesn't mean that perceived offense should be entirely ignored.
Let's imagine a scenario: as in the example above, an English person says "can I burn a fag" and an American person believes they are casually thinking about hurting a gay man. The English person meant no harm; but, the American person is not at fault for not understanding this. They can complain, and the English person should explain their intent. The American person should learn and remember the meaning of this expression in British English, and the English person will need to remember that this expression may sound offensive to American listeners, and avoid it in such scenarios.
Basically, the important aspect is that a misunderstanding is not always the fault of the listener (which the intent-only model suggests: you took offense, but I didn't mean that, so you're silly for feeling offended). It is of course also not only the fault of the speaker (which the perception-only model would suggest: I felt offended by what you said, so it doesn't matter what you meant, you owe me an apology or more). Both parties are involved in a miscommunication, if indeed there was one.