Comment by ttul | Hacker News Reader

Comment by ttul | Hacker News Reader

I read a lot of niggling comments here about whether Claude was really being smart in writing this GIF fuzzer. Of course it was trained on fuzzer source code. Of course it has read every blog post about esoteric boundary conditions in GIF parsers.

But to bring all of those things together and translate the concepts into working Python code is astonishing. We have just forgotten that a year ago, this achievement would have blown our minds.

I recently had to write an email to my kid’s school so that he could get some more support for a learning disability. I fed Claude 3 Opus a copy of his 35 page psychometric testing report along with a couple of his recent report cards and asked it to draft the email for me, making reference to things in the three documents provided. I also suggested it pay special attention to one of the testing results.

The first email draft was ready to send. Sure, I tweaked a thing or two, but this saved me half an hour of digging through dense material written by a psychologist. After verifying that there were no factual errors, I hit “Send.” To me, it’s still magic.

Were yout not concerned about the privacy implications of uploading your child's sensitive halth data to a private LLM?

Not OP, but parent of multiple school-age kids and both:

1. You're 100% right, there are privacy concerns.

2. I don't know if they could possibly be worse than the majority of school districts (including my kids) running directly off of Google's Education system (Chromebooks, Google Docs, Gmail etc.).

Can you opt out? Are there privacy-friendly schools?

Could you enroll your child under a fake name? How messed up would they think that is :D

You can generally register your child under an assumed name - at least, that is possible in my area. Families can choose this option if there is any threat to the security of their child.

It's important to differentiate concern(a feeling) from choosing to upload or not. In the calculus of benefits and risks, The feeling of concern(potentially leaking PII/health information) may be outweighed by the benefit in education. Even if someone is concerned, they may still see the positives outweigh the risks. It's a subjective decision at the end of the day.

I should have clarified that I used Adobe Acrobat to redact his personal identifiers from the report before uploading it to Claude. I generally also prompt using fake names. It's not perfect, but it's better than nothing.

And, on another note, this may be foolish, but I generally trust well funded organizations like Anthropic and OpenAI on the assumption that they have everything to lose if they leak private information from their paid users. Anthropic has a comprehensive and thoughtful privacy policy (https://www.anthropic.com/legal/privacy), which specifies they do not use your data to train their models, other than to refine models used for trust and safety:

"We will not use your Inputs or Outputs to train our models, unless: (1) your conversations are flagged for Trust & Safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Acceptable Use Policy, including training models for use by our Trust and Safety team, consistent with Anthropic’s safety mission), or (2) you’ve explicitly reported the materials to us (for example via our feedback mechanisms), or (3) by otherwise explicitly opting in to training."

As for defending against a data breach, Anthropic hired a former Google engineer, Jason Clinton, as CISO. I couldn't find much information about the relevant experience at Google that may have made him a good candidate for this role, but people with a key role in security at large organizations often don't advertise this fact on their LinkedIn profiles as it makes them a target. Once you're the CISO, the target appears, but that's what the big money is for.

Thanks for the vote of confidence. I led the Chrome Infrastructure Security Team hardening for insider risk and generally defending against APTs for the last 3 years at Google. Before that, I was on the Payments Security Team defending PII and SPII data up and down the stack. Indeed, I and the company take this very seriously. We're racing as fast as we can to defend against the run-of-the-mill opportunistic attackers but also APTs. We've ramped the securtiy team over the last year from 4 to 35 people. I'm still hiring, though!

wink wink

Reply on news.ycombinator.com