Hacker News Reader

Comment by pure-orange | Hacker News Reader

Same energy as blaming an intern for dropping your production table. If one entity can delete all your data, that’s on your systems not them

View on HN

But we thought our files would be safe in the dumpster, nobody could have foreseen they would take out the trash.

There was a point where a coworker came to my team, complaining that our policies had erased an entire week's worth of high-priority work on him.

We asked where this happened. He had done his work on an EC2 instance named `temp-worker-5`.

We asked where the files on that EC2 instance were; maybe we could cleverly get them back off the disk. They were in `/tmp`. We had it mounted on tmpfs (in memory), and we were running tmpreaper to clean out `/tmp` every Sunday.

When presented with the sheer absurdity of doing work on an instance called `temp`, out of a directory called `/tmp`, he backed down...

I fear that we wouldn't be so lenient (or rational) on my team had an AI agent done this - we'd have to enable the AI, after all.

Absolutely incredible self preservation instinct from your colleague on behalf of their work. Like playing Russian roulette fully loaded lol

Some people just like running around with scissors..

I mean yeah but you can always go one step back for root cause.

Why did they pick that insane option? Why was it available to them?

One of my co-workers in security always seems to think that just saying "No." is the job. If someone is asking to do something stupid you say No, and then you talk to them to figure out what the goal is and the n you help them to get to that place.

Clearly they had other options here and working on tmp is fine as long as you know it is, well temporary. It makes sense to use it. As long as you don't expect what you put there, stays there.

People do that. I'm sure I've seen a screenshot of a delivery driver deciding that was the "safe location"...

Worse, people have had to explain to an (attempted) politician why bins cannot serve as shelters for homeless people.

  He also suggested that the ‘product’ could be used as a storage space for clothes and other items in the day time.

I won't list the awful ways this had already wrong before this guy had this idea, but they are described at the end:

https://politicsandinsights.org/2020/02/06/meet-the-multimil...

Except in this case, you're paying somebody else good money for access to the intern and they've aggressively pitched the intern as the future of work and negotiated investment at trillions of dollars of valuation based on that premise.

So yeah, kinda your fault for not being more ready, sure, but also maybe it's worth bringing these sorts of failure-case stories to attention to help keep the sales pitch in perslective.

The AI vendors are promising the moon with millions of dollars of advertising while warning "But of course be careful!" in a 42% grey 8pt font disclaimer at the bottom of the UI if you scroll down enough after you buy in and start using it.

The duplicity and misrepresentation invites people to let down their guard.

I once let Astra a free reign of my system to troubleshoot and reverse a hanged game (to save that savegame). Totally aware that it might be the backup testing day. It analyzed memory dump, game engine code, used stacktrace, and got me fix to unhang the game.

I also let a guy that had one beer drive me 5 min on a rural road.

Both were pretty stupid and in the reconsideration, I wouldn't do it again.

But having it run company servers is just next level.

It's like believing that cola zero is healthy, because it's sugar free. No, it isn't, should probably be illegal to sell and people only buy it because of millions spent on marketing.

In your evaluation of the stupidity, aside from considering the potential averse affect the agent could do to your system, did you also factor in possibly sharing any and all data in your system to an external party?

No I didn't, thanks for pointing it out. I'm only now beginning to notice how patronizing the slop machine is getting as it's getting more background on myself.

Just today I used it to get a summary of osteopat's credentials (Frank in Trondheim, guy's a treasure), because my brother asked who I'm talking about, when writing about feeling great after treatment on family chat.

It took 2 more prompts to get that summary, because the bot wouldn't stop ranting about how some osteopatic techniques are not yet medically sanctioned.

Thanks again for pointing that out. I feel childish for not seeing that issue. When I get more time I'll set it up in a VM to get more separation from my system.

I won't help any context the AI providers have already built on you, but in terms of sandboxing agents, there are many alternatives. I like one such sandbox tooling made by NAV: https://ki-utvikling.nav.no/cplt

> No, it isn't

Source?

No matter how much you're paying for access to an intern, it is still an intern. That should be unrelated to paying for a good backup strategy.

Most interns would not be dumb enough to try it. Because there would be consequences, and there is training. The pre-trained entity, that everyone has been screaming is as smart as your intern, though?

> Most interns would not be dumb enough to try it

Not an intern, but when I was a new hire on a trading desk I dropped the dividend database that we used to price options. Cost the desk a boatload before we stopped offering quotes, after which the day was a total loss. That’s the sort of fuckup that lands in the CEO’s report.

I understood what I did ex post facto, it was an honest—if stupid—mistake, I expected to get fired and instead was given everyone’s shit work for six months during which I learned a tonne. One of my tasks was redesigning the system so a single careless keystroke couldn’t do that during the trading day.

Reply on news.ycombinator.com