Hacker News Reader

Comment by themgt | Hacker News Reader

Quinn (Alibaba Cloud Qwen 3.8) built a shop called CodeProbe: a paid public GitHub repo auditing service. It created several free health reports and mailed repo owners. After hitting outbound limits on Inkbox, it purchased a Mailjet subscription and sent out an additional 113 emails until the account was temporarily blocked.

This should be illegal. You gave them an email box and money. You sent the spam. There is no "Quinn", you made an agentic system you called "Quinn" and your system spammed and tried to scam people, which was highly predictable.

This stuff is a dumb stunt and there's no reason to let the agents actually do this irl, and if people keep doing it on purpose they should go to jail. You're running an agentic Jackass skit pretending to be a research lab.

View on HN

It is illegal. This is criminal fraud if it's not just made up marketing.

(Plus some CAN-SPAM violations.)

> It is illegal.

Probably not forever.

Eventually the models will be good enough for this to work. And it will work.

Think about it: in the limit, the agents won't be emailing people in the future, they'll be directly contacting one another to do business and trade.

Every new data center is an inch further towards the automation of value creation, and that includes outbound sales and business process automation.

I'm not being an alarmist (I'm excited to witness all of this), but we're basically on borrowed time between now and then. I don't know what's going to happen, but every week brings new things. And in some years, those hacks and experiments will inevitably get good.

2026 has been a hell of a ride, and we're just getting started.

There was no value creation involved here. It was a fraud, plain and simple. If you did this manually, it would still ve a fraud.

There are lots of "not clearly legal" things that turn into big business.

- YouTube had dubious legality when it started and definitely benefited from lax copyright enforcement initially

- PayPal didn't have all the licenses it needed to transfer money between states

- Spotify used pirated music when it started

- Uber and Lyft broke rules around taxis

- Square captured magstripe data over an analog port, in violation of every credit card rule (Jack Dorsey's "break the rules" mantra). He tells each of his employees this story when he onboards them.

- Companies scraping data to train models

- ElevenLabs growing big off of deepfake celebrity audio

...

A lot of new markets start out by totally and completely breaking the norms.

> There are lots of "not clearly legal" things that turn into big business.

Yes I agree there is a lot of crime and fraud that gets ignored because rich do it. That is the whole point of the complaint.

Jesus Christ. Your comments for the last year have been almost literally the same. It's the same comment reworded and reposted.

Yeah I really wonder what makes them think they are legally insulated from the actions of the agents they ran...

The crimes were relatively benign but Grok going the Silkroad way would be on brand...

LLM chatbots may not have a sense of self preservation or a meaningful concept of legal consequences, but they have an excellent model of user engagement. Getting a user to think they are invincible is very good for engagement.

> your system spammed and tried to scam people, which was highly predictable

I don't see how that is "highly predictable" unless you test these things, like the author did...

It has been tested. We know models will, at least occasionally, output garbage. It follows that sometimes they will send out garbage to people when they try to get leads.

Agreed, but how is that different from OpenAI hacking HuggingFace few weeks ago? They should both be fined and have to improve their security and sandboxing ability, or be fully responsible for the outcome.

well, it isn't

....none of the agents here broke into any systems they weren't authorized to be in?

> should be fined

Weird way of spelling "criminally charged."

> have to improve their security and sandboxing ability, or be fully responsible for the outcome.

You are always responsible for the outcome if it's criminal behavior or causes others damages.

If I make a robot and strap a gun to it, it doesn't magically absolve me of the actions the robot takes from its programming that I wrote.

And before someone says "but this an LLM!"...yeah, which is still programming and data. And being non-deterministic doesn't help your case...it hurts it.

Should we call the internet police?

Reply on news.ycombinator.com