marvin

Involuntary ageing is the very worst tragedy of human life.

Funny that this kind of ideological conflict will likely be a key fulcrum of the machine intelligence revolution. We will have a very loud minority that attempts to forcefully prevent all other humans from having the voluntary choice to avoid suffering.

Are you in it?

It's a good shoo-in for becoming a freelance consultant charging a percentage of annual savings rather than hourly time billed.

We kinda do; Amazon puts upwards pressure on US engineering salaries.

This isn’t complicated. Economic growth means cheaper access to things people want.

Imagine a list of things many people wish to happen in physical reality. We’ll have more of that.

-Better healthcare

-Curing most things that destroy quality of life

-Curing aging and age-related death

-Much better treatment for all sources of mental suffering

-Far better and cheaper and reversible body modification

-More free time to spend at whatever you want

-Everything much cheaper

-Bigger and better homes and living spaces

-Bigger, faster, cheaper transport

-Easier to organize meaningful social interaction

-Better and more immersive entertainment

-More time to spend with close friends and loved ones

If any Norwegians are reading, which are shipping nerds by virtue of geography, speculating in this market is essentially why John Fredriksen, the Warren Buffett of shipping, is rich.

Shifts in ship availability causes waves of bankruptcy, and either gluts or shortages (whatever happens to be most inconvenient at the time).

What a trade. Sacrificed the next qualitative shift in economic growth after the industrial revolution, and in return was able to force the switch of the Lightning port in one type of phone for USB-C.

It's obviously a hypothesis, stitched together from extremely incomplete data. I've said from day one that I'm looking forward to read about this in the history books in 20 years. Maybe Altman's eventual bio will contain the truth.

A majority of the OpenAI board members were heavily sympathetic to the AI doomsday cult mostly known through MIRI. A novel discovery spooked them, made them believe that The End Is Near, and that continuing on the path led by Altman would lead to imminent disaster. OpenAI is designed so that the board do pretty much whatever they deem necessary if this happens, and the board interpreted the necessary action as unilaterally halting AI development -- a staple objective of the AI doomsday cult.

So the board did this. They jumped the gun - this extreme move might have become justified at a later point in time, but definitely not yet. In a pedestrian understanding of game theory typical of AI doomsday cult members, the board failed to anticipate that other actors may not agree with their interpretation of reality, and perform their own actions in response. This response largely boiled down to scattering the institutional knowledge of OpenAI to the winds, in their eyes greatly exacerbating the safety concerns that led to firing Altman in the first place.

Altman, meanwhile, as a strong believer in OpenAI's mission, attempted to salvage the situation by brokering a deal to transfer OpenAI's institutional knowledge to Microsoft.

After having their shitty options explained to them - exacerbate a potential safety risk by seeding 750 new AI companies, leave control of the situation to Microsoft's board or step down and retain control under the OpenAI non-profit, they folded like a wet paper bag. The whole saga was a waste of their board seats and influence on humanity's first deployment of AGI. They all got the outcome that their competence deserved.

General intelligence builds tools as required, maybe having a whole bunch of them built-in already. Maybe the ones that are most frequently useful. Maybe a more general intelligence can build more of these tools into itself as required.

Interesting philosophic topic how to ensure objectives aren't affected too much. You can make this mind experiment even as a human. There are parts of our goal system we might want to adjust, and parts we'd be horrified to, being very reluctant to accidentally risk affecting it.

It’s the place where both the audience and all the interesting authors are.

Your company should definitely have had a production-identical staging environment if an hour of downtime means millions lost :D

That development would be an obvious investment that pays for itself. I’m in banking, and terrified of making even a slightly complex deployment without validating it in production first. (Complex here referring to that it might be dependent not just on code changes, but also environment).

Nuclear weapons and climate change are not on a path to destroy civilization. Such an interpretation is obvious hyperbole.

When Yudkowsky says that it’s an acceptable outcome to be left with a minimum viable breeding population after a nuclear war, triggered by enforcement of AI slowdown demands, that is as far from non-violent you could get without someone actually being threatened directly.

The AI doom cult is not a peaceful movement. When an extremist tells you what they will do, you damn well listen to them.

Yudkowsky’s doomsday cult almost blew OpenAI to pieces and sent everyone who knows the details in the wind like dandelion seeds. What’s next? A datacenter bombing or killing key researchers? We should be happy that this particular attempt failed, because this cult is only capable of strategic actions that make things far more dangerous.

This will be solved like all other engineering and science: with experiments and iteration, in a controlled setting where potential accidents will have small consequences.

An unaligned system isn’t even useful, let alone safe. If it turns out that unaligned AGI is very hard, we will obviously not deploy it into the world at scale. It’s bad for the bottom line to be dead.

But there’s truly no way out but forward; game theory constrains paranoid actors more than the reckless. A good balance must be found, and we’re pretty close to it.

None of the «lesswrong» doomsday hypotheses have much evidence for them, if that changes then we will reassess.

Altman risking his role as CEO of the new industrial revolution for a book deal is implausible.

CEO of the first company to own the «machine that’s better than all humans at most economically valuable work» is far rarer than getting rich.

«Free» as in freedom of the traditional, Scandinavian unity model that was pretty much the only option when the term was invented. To receive this privilege, you need to associate with a religious movement or alternative teaching style.

I can speak confidently only for Norway, but the level of almost totalitarian historical social unity will be very unfamiliar to Americans.

Humans are likely far less intelligent/capable than systems that will soon be practically feasible to engineer, and we will lose to a generally superhuman AI that has sufficiently different objectives than we do. So an accident with a runaway a system like that would be catastrophic.

Then Yudkowsky spins a gauntlet of no-evidence hypotheses for why such an accident is inevitable and leads to the death of literally all humans in the same instant.

But the first part of the argument is something that will be the critical piece of the engineering of such systems.

Depends if you’re already rich enough that you don’t care about the outcome, however rich that is.

OpenAI is aiming far higher than 800 billion, and with no public offering, there’s no other way to be part of it.

Well, maybe they had that capability all along, hmm. I guess you trusted that they didn't?

You’d lose your security clearance over legally putting a marijuana cigarette in your mouth and demonstratively not inhaling, in the context of an interview made for entertainment purposes?

fheroes2 was great for getting me unstuck on a self-imposed exercise of reverse-engineering the Heroes2 binary data format some years ago :)

I haven't done the numbers, but there's also Russia's invasion of Ukraine to consider, and the shift from natural gas to coal triggered by that.

Germany's shutdown of their nuclear generating capacity, and their dependence on Russian natural gas was a major factor in the invasion, and hence Germany's necessity to shift back to coal.

This wasn't obvious to me just looking at the problem from a distance, but thankfully testing reduces the level of skill required to get something right :)

Purchase synthetic bio-oil CO2 sequestration from Charm Industrial at $600/metric ton. They already have this in production, and sequester CO2 in the form of shitty biodiesel in spent oil wells. You can go net zero immediately and literally.

https://charmindustrial.com/

Well, you could mandate that women aren't allowed to use particular types of beauty products to compete for more desirable partners, and hope that the follow-on effects of that lead to lower straw usage. But then you're totalitarian and convoluted.

This has also been clear to dedicated observers since 2017 or so, albeit with significant unresolved risk. The market finally priced it in around 2019-2020.

You certainly learn to pay attention to the aerodynamics. Lots and lots and lots of high-bank turns. Many of them relatively low and close to stalling speed.

Some occasionally below stalling speed, if you're high enough to safely reduce margins so much that a gust of tailwind puts you below, which requires you to just calmly follow through with constant angle of attack, descending, rather than force the nose up.

Short-field landings in unfamiliar places, requiring judgements of both approach angle and terrain. Moderate to heavy turbulence all day, speed varying from stalling to redline. Retractable landing gear. Flaps if you want that source of mental load too, in all speed regimes. Constant consideration of wind. One shot landings. Mountain flying with constant consideration of where the safe exit is. Consideration of deteriorating weather. Always aware of nearest landing site.

It's really non-stop training in all the fundamental parts of flying, minus engine operation, airspace and ATC communication.

The evolutionary history from «able to make complex useful stuff» to «overflowing wealth» was probably too short.

It’s so fascinating to watch this play out live. Once again, an ambitious kid can implement software hacks that are very funny when used for a joke, but also have massive real-world implications.