snewman

Have you run the numbers on this, or are you just imagining? In the 70s, solar and (to a lesser but still substantial extent) wind were still extremely expensive, impractical for large scale use. As were the batteries that are needed to smooth out periods of low renewable generation.

I'm an absolute supporter of renewables, and I'd love to see support for fossil fuels phased out, but it's been a long climb to reach the point where that's feasible.

The board's motivation may have been good, they may be "in the right" here (it depends on your view on AI risk, and also on a lot of details of what was going on that we still don't know). But unambiguously they executed abysmally. There is little to be proud of here. OpenAI may have needed to be reined in, and they have probably succeeded in accomplishing that, but they have badly damaged the company in the process, and catastrophically damaged their own credibility, thus reducing their ability to influence events in the future and plausibly damaging the credibility of the entire idea of responsible development. (To be clear, I believe responsible development is important and I am sad to see it discredited.)

Edit to add: if you believe the rapid commercialization of frontier models at OpenAI was a cancer that needed to be destroyed, the likely outcome of the board's clumsy handling is that it probably now metastasizes.

Help me understand, what was gained by removing "co-founder and CEO"? The fact that readers can figure it out is not actually a statement of a benefit, it's merely an argument that the loss is bounded.

Blowback was inevitable but the magnitude was not. The board absolutely bungled this, full stop. Maybe Altman had to go, maybe he didn't – I don't claim to understand the backstory – but the way the board handled and is handling this was extremely clumsy at best.

For renewables, most of the cost is up front, then they're cheap to run because they don't need fuel. (High capex, low opex.) Fossil fuel plants have more of a mix of upfront and operating costs. High interest rates make it expensive to borrow the upfront cost of renewables.

And to be clear (for anyone who doesn't know, I presume parent does and simply didn't think it necessary to explain), this "carbon neutrality" argument is entirely bogus, because one molecule of methane has far more climate impact than one molecule of carbon dioxide, even though they each contain a single carbon atom (CH₄ vs. CO₂).

Apparently, the underlying science is basically that "wigglier" molecules have more propensity to intercept photons, and that's why molecules like methane and refrigerants have more impact than the relatively simple CO₂.

Good grief!

At a guess, it is copy/paste from MATLAB marketing materials, intended as a sort of appeal to authority – "we used MATLAB so our results must be good".

Bear in mind that grid electricity is getting cleaner over time. We need to skate to where the puck is going, which is 100% clean electricity. Now is an excellent time to be developing and preparing to scale technologies that work well with clean electricity.

I'm honestly not sure what your point is. Are you saying that, if there was less cold quantitative analysis of the global CO2 potential from forest management, people would be planting more trees in their local neighborhoods?

Absolutely, trees can provide many benefits separate from CO2 absorption. I was simply responding to GravityLab's comment about CO2. However, it's also worth noting that planting the wrong trees or in the wrong place can backfire: displacing more appropriate vegetation; fire risk; etc.

Planting more trees (and, more important, preserving existing trees) is an important tool in the toolkit, but shouldn't be overestimated, and I don't think the word "easily" is warranted.

Project Drawdown estimates that 3.2 to 4.96 Gt CO₂/y could be sequestered (through 2050) using a combination of tropical forest restoration, temperate forest restoration, and tree plantations on degraded land. The upper figure is only about 10% of worldwide emissions: significant, but not a game-changer. And the projects involved would be nontrivial, to say the least.

If you're interested, I dug into this topic in some depth in a blog post a little while back: https://climateer.substack.com/p/forestation-2.

Related: robotics pioneer Rodney Brooks just wrote up an incident where a Cruise cab he was riding in stopped suddenly in the middle of an intersection, nearly causing a bad collision: http://rodneybrooks.com/autonomous-vehicles-2023-part-ii/. Consistent with the discussion here of Cruise cars being dangerously timid.

The development of better power technologies is ~necessary, but not sufficient, to eliminate coal (and other carbon-emitting energy sources). There are all sorts of friction effects which can delay a migration to clean power even when the economics are favorable, so direct efforts to push the decision forward are an important piece of the puzzle.

[author here]

> And AGI doesn't need to completely destroy the earth to be really bad for humans. Just taking over a lot of the resources we need would do the trick.

Agreed. I am not arguing that AI could not become superhuman, or could not overwhelm us. I'm merely arguing that:

1. It is not guaranteed that mildly superhuman AGI would inexorably lead to a runaway feedback loop of capabilities increase. I'd agree that it's possible, but I often see statements that it is inevitable, because an AI smarter than us would be able to create an AI smarter than we can. Such statements fail to take into account that the sequence of successive AIs might converge (at mildly superhuman) rather than diverging (to a singularity).

2. Even if AI capabilities diverge, there's no guarantee that would happen quickly ("foom"), because even as AI capabilities increase, the effort needed to achieve each further increment in capability will almost certainly also be increasing.

I do take the potential for superintelligence very seriously; see, for instance, https://amistrongeryet.substack.com/p/get-ready-for-ai-to-ou.... But I also think that some current discussion is going overboard in the other direction, and waving away likely hurdles on the path to superintelligence (see https://amistrongeryet.substack.com/p/the-ai-progress-parado...).

This excellent tale has appeared many times on HN; here's dang, in 2021, listing some of the past threads:

https://news.ycombinator.com/item?id=29213472

Unfortunately you can't really rely on this sort of comparison between numbers obtained from different sources, except in the most unambiguous circumstances, which rarely arise for a complex topic such as sector emissions and cost of mitigation. The 1.5% and 2.4% figures may use different methodologies. The total-cost-to-decarbonize estimate probably entails a variety of unrealistic assumptions; it's hard to take into account future improvements in technology, all of the different ways the market will find to minimize costs, etc.

If you're interested, I discuss the general danger of working with such quick-grab numbers in https://climateer.substack.com/p/numbers.

In this scenario, it's nothing to do with the cloud provider. Migrating a live production system is inherently difficult. You can make a reasonable analogy to moving houses – say, with two kids who are in school, and while you and your spouse are both working. No matter what guarantees you're given regarding the condition of the new house, simply moving all of your stuff (while you are using it) is a big hassle.

It's not so much that EC2 Classic offered any features that were difficult to live without. It's just that migraine away from it means migrating, period. You need to move all of your systems, including any data stored on those systems (in instance store or EBS), to effectively a new data center. Migrating a live production environment can be a pain and/or cause downtime.

Actually, I believe the claim in the article is precisely that spending more time leads to better posts and that this leads to more subscribers. Basically, the author feels that he achieves more (in readership growth, but also on other dimensions he cares about) by taking the time to write one really good post instead of using that same time to write two mediocre posts; he contrasts this with the standard advice that publishing frequently is critical to building an audience.

If you have very wide error bars, shouldn't your estimate of impact probability be much lower than 3%? Most trajectories within your error bars will not intersect the Earth.

Actually the definition of 'grok' is discussed in the book; you can find some relevant snippets at https://en.m.wikipedia.org/wiki/Grok. My recollection is that the book says the original / literal meaning is "drink", but this isn't supported by the Wikipedia quotes and perhaps I am misremembering, it has been a long time.

Yes, although a better way of saying it is that it is not the helium which has lifting power, it is the balloon. A balloon has lifting power equal to the weight of the air displaced by the volume of the balloon. From this, you must subtract the weight of the balloon and its contents. Helium is one of the few materials that allows you to wind up with a positive net lift.

(Also, for this to work, 1kg of helium would occupy a balloon much larger than 10cm^3).

I maintain two, with pretty different content. Both are intended to explore deep topics (AI and climate change, respectively), aiming for a middle ground between academic papers (meaty, but often hard to understand + contextualize if you're not already an expert) and popular press (often over-simplified or off the mark).

https://amistrongeryet.substack.com/ – was intended to be a general "things I've learned after coding for 40+ years", but so far just chronicles my attempt to wrap my head around the actual capabilities of current AI models and the potential trajectory and impact on society.

https://climateer.substack.com/ – my attempt to explain some of the big / controversial topics in climate change mitigation.

Regarding opportunity cost, it depends on what your limiting factors are. For instance, to the extent that solar installations in the US are currently limited by grid interconnect, this is a good way to deploy additional solar in service of emissions reduction.

Also, in the medium run, we are going to need to scale hydrogen and e-fuels and this is a good way to start climbing that learning curve.

Great post. I'll propose one more "myth": writing a blog doesn't benefit you personally, except by drawing attention. I've found that writing about a topic can also be a great way to solidify your own thinking on that topic. There ideas that I've understood much better after blogging about them. Basically, you're using the entire Internet as your "rubber duck".

So much negativity on this thread. I think it's at least worth mentioning that the paper does present many genuinely interesting ideas; for instance:

1. The change history of a repository contains more information than just the final state of the code. The change log constitutes useful training data which can allow a model to learn things it might not get from just being trained on completed code.

2. Software engineering consists of a lot more than just writing code. They've been able to train a model to do other tasks, such as reviewing code and responding to review comments.

3. Operations such as renaming a variable can be modeled as a single high-level action, rather than as a set of raw code edits. A model trained to generate such higher-level actions can potentially achieve better results. "For example, a rename might touch a file in dozens of places, but a model can predict a single rename action."

4. A developer's recent activity can be used to help predict next steps. They give the example that if you add a parameter to a function and then move the cursor into the function documentation, the model can predict that your next step is to document the new parameter. "Without history, the model wouldn’t know whether the missing function parameter is intentional (because the developer is in the process of a longer edit to remove it) or accidental (in which case the model should re-add it to fix the problem)."

A longer example:

> ...we started with a blank file and asked the model to successively predict what edits would come next until it had written a full code file. The astonishing part is that the model developed code in a step-by-step way that would seem natural to a developer: It started by first creating a fully working skeleton with imports, flags, and a basic main function. It then incrementally added new functionality, like reading from a file and writing results, and added functionality to filter out some lines based on a user-provided regular expression, which required changes across the file, like adding new flags.

This ability to emulate a natural sequence of authorship steps to create a new code file seems similar to "reason step by step", but specialized for coding.

This is oversimplified at best. Not all VCs are created equal. A broad-brush characterization like this removes any incentive for VCs to behave well (if everyone will assume the worst of you, why bother trying to build a good reputation), and discourages entrepreneurs from vetting their investors (why bother, if they're all equally bad).

I think the value here is around starting with a known default. If someone hands you a contract which consists of a known boilerplate plus three changes, then (assuming you're already familiar with the boilerplate) you don't need to carefully read the entire contract word for word, you only need to look at the three changes, plus whatever customizations you'd like to propose. For the vast majority of the terms where neither party feels a need to deviate from the standard, you get two benefits: (1) the drafting party's lawyer won't put in onerous terms just to see whether they can get away with it, and (2) you don't need to read it carefully, because you already know what it says.

(This assumes some mechanism for ensuring that what is claimed to be a copy of the standard boilerplate, is in fact such, and has not been sneakily modified.)

You are correct that water vapor is the primary greenhouse gas. Despite that, water vapor emissions are not generally considered to impact warming. I think that is because the water content of the atmosphere is already in balance due to the massive amount of evaporation and condensation (e.g. rain) that is constantly going on. If we dump some extra water vapor into the atmosphere, that will balance out as a bit of extra rain and/or a bit less evaporation somewhere else; atmospheric water vapor levels won't change.

I don't know whether a large scale, ongoing intervention, such as permanently flooding a large area, could nudge this balance sufficiently to result in a sustained (if small) increase in atmospheric water vapor content, and thus an increase in warming. An interesting question!

I wrote about the role of water vapor in global temperatures a while back at https://climateer.substack.com/i/60052576/you-thought-the-ma.... My primary source was an IPCC report, specifically the sidebar on pages 8 and 9 of this PDF: https://www.ipcc.ch/site/assets/uploads/2018/02/WG1AR5_Chapt....

[author here]

It's true that I didn't present a lot of concrete evidence of GPT-4's limitations. This is a blog post, not an academic paper.

To my mind, the most concerning thing about GPT-4's performance on these two examples is not that it got the wrong answer, but that it utterly, utterly failed to understand that it was having difficulty. Even after repeated hints and prompts, it continues to make exactly the same mistakes, with no attempt to reason more carefully. There are plenty of other examples around of GPT-4 (to say nothing of earlier releases) having similar troubles.

If you scroll down to the Further Reading section at the very end of the post, you'll find a reference to an extensive paper from Microsoft Research that comes to similar conclusions regarding GPT-4's limitations (I found this only after writing the initial draft of the post). For instance:

> These examples illustrate some of the limitations of the next-word prediction paradigm, which manifest as the model’s lack of planning, working memory, ability to backtrack, and reasoning abilities. The model relies on a local and greedy process of generating the next word, without any global or deep understanding of the task or the output. Thus, the model is good at producing fluent and coherent texts, but has limitations with regards to solving complex or creative problems which cannot be approached in a sequential manner.

The main point I tried to make is that GPT-4's "nearly superhuman" performance on a wide variety of tasks is somewhat illusory, and leans heavily on memorization. I spelled out some reasons why I think it looks more intelligent than it is. Relative to past work in AI, it is extremely impressive. Relative to the threshold required to perform economically useful work, it's... mixed; we're already seeing useful applications, but I think the majority of "information worker" tasks are still beyond it, and I'll go ahead and predict that the same statement will hold for GPT-5.

I don't think this sort of dismissal adds anything to the conversation. I myself, in the act of typing this comment, am "generating text".

I think it's interesting to discuss the capabilities of leading-edge language models like GPT-4, because (a) they are already exhibiting the ability to perform a wide variety of useful tasks, and (b) it's clear that there is still a lot of unrealized potential here.

Can you clarify the implications you see here? Are you saying that these LLMs are somehow uninteresting or incapable? That there are limits to what they will be able to accomplish even with further improvements? Or something else?