sarchertech

I’ve worked on HR software. There are currently at least 3 large scale, bespoke, homegrown payroll/HR systems in use by the Federal government. There used to be many more. There are many reasons to outsource software but the government can’t hire enough engineers is not one of them. It’s clearly been done before.

>Just customizing something like PeopleSoft to fit is a large task in itself, let alone trying to implement bespoke software

Customizing large enterprise software is more like writing bespoke software than it is what most people think of when you use the word customization. You end up paying enormous vendor margins on top of the costs for something that is effectively bespoke software.

There is so much bespoke software out there that is at least as complex as government payroll software.

Enterprise software is complicated mostly because of number of customers it can support.

Bespoke software can be orders of magnitude less complex. There are many reasons companies choose to use vendor solutions, but for large organizations it’s usually not “we literally can’t hire enough engineers to build it.”

There are so many counter examples.

Last time I looked the other app stores didn’t show anywhere near that much of a bump (that was a several months ago though).

And a few months ago I went through a random sample of new apps on the App Store and the vast majority were just wrappers for AI APIs. So it was more of a new gold rush situation than a productivity bump.

There was much more than just the high profile egregious cases though.

Most food adulteration is a quality issue more than a safety problem.

Buy old peas but put a little green coloring in is almost the exact same thinking as let’s replace the metal gears with plastic ones to save a dollar per item. The vast majority of companies doing that didn’t kill or seriously injure anyone.

Companies have always been doing this.

2 things.

1. If you turn the temperature down too far, the output is just bad and no amount of running prompts optimization will let you hill climb your way to good performance.

2. It’s not about determinism vs non-determinism. It’s about chaos. A perfectly deterministic model is still chaotic. Meaning that very small changes to the input result in very large changes to the output.

Turning temperature down doesn’t actually get you predictable or reproducible behavior across different inputs.

I don’t know if that’s true. Look at the amount of lethal product defects over the last 100 years.

And before the FDA existed, companies were perfectly willing to put formaldehyde in milk, mercury in candy, and arsenic in vegetables.

If turn the temperature down for tests, they won’t match production behaviors. If you turn it down too far in production, the output will just be bad.

Yeah but in my experience AI boosts output of those humans 10x and only boosts output of programmers who do write maintainable code 50-100%.

This isn’t a web UI, it doesn’t need icons.

But more importantly, if you aren’t the kind of person who could make a halfway decent text only poster before, you are going to make the posters at the top of the article. Those are objectively harder to read than something typed up in word.

You aren’t going to know to tell the AI to “ Go for a clean, unfussy, bright layout with a bold striking spring-themed graphic. Avoid pastel/airbrush/oil style art or images of people.”

That’s an assertion with no evidence that doesn’t match my experience.

Agents require far stricter guardrails than humans. Without linters, tests, static analysis, oracles etc… no agent can create a large program.

Even if you’re correct, you just build those checks into CI so that neither humans nor agents can skip them.

I said

“You can’t deterministically keep them from making even a tiny fraction of all the possible errors they can and do make though.”

And you replied

“This may be platform dependent.”

I’m unsure how else to read that other than an implication that this might be possible on some platforms.

> better code than a human would write,

Why would that be the case? You can run human written code through the same “linters, compilers, static analysis, fuzzing, testing” as you do agent produced code.

Code analysis can’t detect a sizable fraction of all possible errors because for any Turing complete language the number of possible errors is infinite.

You can’t deterministically keep them from making even a tiny fraction of all the possible errors they can and do make though.

You can’t keep humans from making those errors either but you also don’t let an error prone human crank out 20k LOC per day without forcing other humans to understand it.

> at least no one would be stupid enough to do the final assembly of anything that might be a weapon from parts the AI designed and arranged to be made.

Worrying that an AI is going to figure out how to make a device that looks to everyone investigating it like a power generator, but once turned on kills all humans, is like worrying that an AI might discover magic. I can’t prove it’s not possible, but I’m not spending time worrying about it.

As weapons get more and more powerful they have always gotten more and more complex and required more and more time, space, money, and infrastructure to build and deploy. There’s no reason to think that this will change with AI.

Maybe don’t accuse me of sexism because of a feeling you got based on informal sample you took of writing from some other people on a website.

I read the report you linked and it is purely speciation. If you’ve ever read any von misses, this report sounds exactly like his writing in that the important predictions of are entirely asserted rather than derived.

As for the background of the author. a BS, a few years as a “researcher” at a non-profit and absolutely zero peer reviewed publications reads more like the resume of a tech journalist than a serious researcher.

By traveling here Aliens have demonstrated the ability to actually manufacture technology more advanced than ours. Even their method of transportation would be an existential threat.

Inventing the concept of new weapons isn’t the same as building them at a scale capable of wiping us out. Cobbling together a human vaporizing ray out of toaster parts is pure fiction. You might as well worry about AI developing the ability to perform magic spells.

In reality any new weaponry takes time, space, and effort to build. The idea of an AI building secret factory in the jungle producing indestructible flying death cubes is the tech bro version of preppers who the woke up every morning worrying that Obama was going to take over and enact sharia law.

> Ajeya Cotra

Please look up her resume. If that’s one of the people who are “much smarter than you”, you’re either very gullible or you don’t give yourself much credit.

She has a BS and and has essentially never worked in a technical role. Her primary role over a short work history seems to have been more focused on getting funding to study AI safety.

There’s nothing revelatory in that article it’s just breathless speculation.

That’s true. But this idea was unworkable because of basic biology/physics. It doesn’t matter how good the sensor is if you’re dealing with bad samples.

For most of the tests they wanted to perform, a finger stick will always provide an unrepresentative sample. And not in a predictable way.

The thing is experienced people knew this. All you had to do was to talk to anyone who knew the biology and they would have told you this could never work. Not this can’t work with today’s technology, but this is fundamentally at odds with reality.

Yeah but the “vision” was just the ideas of a kid with 2 semesters of college classes under her belt saying “what if”. It was never going to work. She had tons of competent engineers working for her. The basic idea was and is unworkable.

If she had come up with an actual good idea with only 2 semesters of experience then sure you could say she was a genius. Even if she couldn’t successfully build a company. But the most you say is that she is a really good conman.

If you think we’re 10-15 years away from self replicating drones, I don’t think there’s anything more I can say to you.

AI research takes physical space and physical prototypes. Even if you can simulate a lot, the simulation is never going to be accurate enough to go from simulation to perfectly working prototype. Especially not with a single rack of GPU servers.

But let’s say that a rogue AI already had plans for a perfect killbot, setting up a factory to build them takes time, iteration, space, and manpower as well.

And you would have to do all this undetected.

It’s possible to imagine technology that would allow a rogue AI to take over the world, but we’re nowhere near that yet. We may never get there.

Missiles have been doing this for decades.

AI drones can’t wipe out humanity without being able to replicate.

AI could mostly destroy civilization if you gave it sole launch control of ICBMs. It could also cause a lot of damage to society with no physical presence.

But realistically we’re nowhere near AI powered robots being an existential threat.

If you can prove that there’s an imminent threat, you can get an injunction.

In practice, writing programs this way tends to result in programs that use less memory not more.

I think I read it as a bit more flippant than you meant it because it was worded assuming the answer was yes.

I mean sure don’t leave a garden hose running overnight. That’s insane. If you leave it running next your house you’ll flood your basement.

You said you had a pool though. That and trying to keep 2 non native trees alive in the desert is probably equivalent to irrigating a large yard.

OP confirmed it was gasoline.

Even if you’re driving a Prius and getting 50mpg, if you’re “driving all the time” you’re spending at least $100 a month in gas. There’s nowhere in the country where a water bill is $1000 a month unless you’re doing a lot of irrigation.

The only way I can read what they wrote to mean that is if they are saying water costs $30-40 a gallon.