Is it already a cliché to cite Idiocracy?
unless frontier models (or whatever employs in medicine) are being kind enough to absorb the lawsuits here, I don't see how this ends well.
The accountability is with the company owners. Like with every healthcare startup, there's no reason to assume they'll love being sued or fined so much that they will bake violating HIPAA into their business plan.
They have lawsuits built into this, and either by hubris or bribes believe this is the most profitable route. "Its just acne, what could possibly go wrong?"
I don't think it's automatically true though. In the case of acne medication, I remember there's a well-known one that carries the very real caveat that it'll kill or severely deform the baby if you're pregnant. When you consider the old classic line "Think of the average person. Half of the people are dumber than that" I don't feel comfortable with that medication sitting on the shelf next to the Clearasil, where a busy person will just grab it because they heard it was super effective, or because the other types were out of stock.
The AI inference involved there no doubt costs pennies; so having it screen the patient to ensure that she understands the risks and has taken a pregnancy test recently if unsure, seems like a "why not" kind of thing.
Now, if some AI-healthcare company is capturing 90% of the money saved by not having doctors do this, then it would be barely worth it.
[0] - https://www.mobihealthnews.com/news/nolla-health-launches-ut...