The frustrating (and silly) thing is that this argument is used a lot to attack left-leaning folks who _do_ engage with many people whose experience and world view are very different from them... like people who are homeless, immigrants from other countries, people who are racially minoritized, people who are disabled.
For many people who don't experience those kinds of life experiences, building relationships with those folks can be really tough and bring into question a lot of the foundations of their world view.
The argument that left-leaning people won't engage with right-leaning people often feels like it's used as an excuse for right-leaning folks to use rhetoric and hold positions that routinely disenfranchise and threaten the safety of the kind of people that left-leaning people have worked to empathize with and build relationships without consequence. That the people who continue to have right-leaning views don't seem interested in putting in the same _effort_ to empathize and build relationships with people other than themselves is both hypocritical and not surprising to me.
Finally, engaging with "challenging" opinions is all well and good as a mental exercise, but building and maintaining a relationship with someone is a project that requires continuous work (even as just a friendship) and I think it's worthwhile to be selective in the people who you put in that kind of work for.