This article feels “ripped from today’s headlines” for me, as my mother-in-law was recently in the ICU after a fall that caused head trauma. The level of AI-driven automated decision-making is unsettling, especially as it seems to allow large organizations to deflect accountability—“See? The AI made us do it!” I’m not entirely sure what guided her care—or lack thereof—but, as someone who frequently works in healthcare IT, I see these issues raised all the time.

On the other hand, having access to my own “AI” was incredibly helpful during her incident. While in the ICU, speaking with her doctors, I used ChatGPT and Claude to become a better advocate for her by asking more informed questions. I could even take pictures of the monitors tracking her vitals, and ChatGPT helped me interpret the readings, which was surprisingly useful.

In this “AI-first” world we’re heading into, individuals need their own tools to navigate the asymmetric power dynamic with large organizations. I wonder how long it will be until these public AI models get “tweaked” to limit their effectiveness in helping us question “the man.”