I'd say the bigger privacy concern is that those chat histories are not just stored on your device - they are stored by the AI platforms as well. I think we've learned our lesson from social media that the platforms will store and use your data for their gain, and your pain. Maybe not today, but over the next few years/decade, as they monetize their platforms ever more?
So I agree that privacy concerns are legit... but this article is looking at the small potatoes where there is a much more terrifying big picture.