Though it doesn't entirely answer your question, several philosophers of speech have identified no metaphysical difference between speech and non-speech, and some find the justifications for a law guaranteeing freedom of speech to be ultimately unfounded[0]. The issue for them is not whether speech causes harm (it does) but whether the harms are sufficient to allow some kind of regulation on speech; there is no evidence showing that speech "invariably causes less harm [than non-speech]". Several have argued this to be the case in the context of hate speech or pornography - it's conceivable their arguments may extend to widening current law (or college policy, since most discussion on the topic takes place in the context of universities[1]) to take into account microaggressions - which, as the researchers have noted, may not always be conscious[2].

[0] https://www.law.berkeley.edu/wp-content/uploads/2018/11/Bris...

[1] https://www.jstor.org/stable/2696265?seq=1#page_scan_tab_con...

[2] Sue, Derald Wing. Microaggressions and marginality: Manifestation, dynamics, and impact. John Wiley & Sons, 2010.