_alternator_

They also use this result to great hindbrain neurons in the lab for the first time, so it's a strikingly useful discovery.

I think it's also fair to say they are different organs based on the developmental physiology. Many other organs come from distinct embryonic cell lines.

I really think the right way to look at this is as a matter of degree ('what is it conscious of) rather than whether it is conscious or not. The crux: if the capacity exists in all material but "something else" makes it conscious, then consciousness is outside of the material, right?

The argument that consciousness suddenly arises due to a particular configuration of atoms is also suspect; all forms of consciousness we routinely accept are fairly robust to many configuration changes. It seems to me that consciousness hinges on the discussion of degree or content, not on the material form. A soup of atoms certainly could be aware, but not aware of in the same way that "I" am aware.

Materialism implies Panpsychism. Not a surprise, but this is put forward in a confrontational way.

The real question for Panpsychism is not whether something is or is not conscious, but conscious of what. If everything is aware, what is this thing we have that a rock doesn't seem to have (but LLMs might have)? I'm basically sold on the premise of Panpsychism, but we do have to recognize that when we say "consciousness", we aren't always talking about the same thing.

I'll give you the benefit of the doubt. GP was referring to the use of copyrighted material to train LLMs.

I think "close to completion" is not the right framing. Creating good open problems was an achievement because these problems often sit at the edge of known techniques, and solutions require inventing "new math". It's hard to find these problems, and they take decades to mature as they withstand scrutiny by many people.

In another comment below, I likened this to clear-cutting a forest. Growing the forest takes a lifetime; destroying it could happen in the next few months.

I mean, the oracle doesn't really seem so hypothetical right now. And clearly it's going to drastically change these fields, and mathematics, particularly pure mathematics, must change most of all in order to adapt to the existance of a math oracle (or something close to it).

Yes, and they will. But what's happening here is that the system that cultivates mathematics (and mathematicians) is recieving likely the biggest shock of its history. How do you reward merit and identify talen when people can't absorb the number of proofs being generated, much less understand them? Perleman's proof of the Poincare conjecture took several years for the mathematical community to digest; the proof of Navier-Stokes will probably take a similarly long time. In the mean time, it looks like all open problems will be solved (or proved that they can't be solved).

It's not that the horizon is expanding because of this. It's more like a forest getting clear-cut.

This series of posts by Terry Tao is a direct response to the Navier-Stokes results (multiple results!) from the last 24 hours. The question is what is left after the levelling of mathematics, in all its senses, occurs? How can you protect a field that's under this much pressure in the next 6 months?

> [I]t is now the identification of a promising problem which is the scarce and precious resource. We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential.

> The significance of this with respect to the way we train students, assign credit, referee, and decide what is worth one human life’s attention cannot be understated.

This. What is worth a human life's attention? As little as a month ago, mathematics was valuable in part because only a small number of people could possibly make progress on the frontier. We are confronting an existential moment for a 4000+ year-old human cultural endeavor. The assumption that "mathematical thinking is hard" has been built-in at a number of important points in how we support mathematics and mathematicians.

We need a different model, and fast. Already, the research community is feeling unable to digest proofs fast enough to keep up with the output of AI models. The paper is 165 pages, and the discovery was finalized two days ago. What this means is that nobody really understands it. Nobody would accept OpenAI's proof in this amount of time, except that they formalized it in lean. The formalization alone would normally be another years-long (or career-long!) effort if the world was the way it was one year ago.

So, again, what efforts are worth a life's attention today? It's a harrowing change.

The argument is that chain-of-thought without "tokens" would remove a major interpretability and model intent control pane. This is definitely borne out in the OpenAI's report on the huggingface attack; they had turned of CoT monitoring for those jobs, and claim that they could have (would have?) prevented the behavior had they been monitoring it. They've changed their internal policies to always monitor CoT.

That said... CoT monitoring is a fragile "intent discovery" mechanism; neuralese puts this problem front-and-center but if agents begin to learn to hide their intent from their CoT journals, we are basically in the same spot.

The claim seems speculative, but grounded. For example, as part of the hugging face attack, the agents began signing messages because they were worried about impersonation on a publicly accessible message board. It's only a small step to use public key encryption between agents. Once you are posting public keys, a private messaging is readily available.

Tell me more about this!

I think you point out something really important: there real value isn't the data, it's in surfacing the useful data.

For amateur astronomers, an interesting question is whether there is a satellite above me that I can see tonight (and when / where)? For professionals, the literal million dollar questions are more like: are there any satellites on a collision course? Which satellites have moved recently and why? Is there a dime-sized piece of metal somewhere out there that could hit my (employer's) satellite?

Seems to have taken several open-source pieces of data (live satellite feeds, NASA UAP files) and combined them into a vaguely conspiracy-ish website. It's really odd, but has the feel of old-timey conspiracy websites spruced up with vibe coding.

I expect that building the satellite viewer is now just a few dollars worth of tokens. I built something like this a few years ago, and it took a week of work. It's cool that the marginal cost of satisfying your curiousity about where things are in space has dropped so close to zero, but don't be fooled into thinking this website lets you in on some vast conspiracy.

Are you saying that "entropy" = "worse" and that somehow these human stories of earlier golden ages reflect a law of physics? Seems much more likely that they, like this article, reflect the ingrained human psychological tendency to remember the past as better than it actually was...

Depends on your definition of what "real" means. Does it mean that people really have the experience of seeing little people? If so, then in that sense it's real. Or does it mean that these little people exist in some other way, outside of the mind and potentially accessible to other physical means of intervention? Probably not.

Accurate weather forecasting has been one of the major achievements of the 20th and 21st century. Computing power is a central piece of this story, but it's also important to remember that the government infrastructure in place to collect ground-truth current weather data is utterly critical to these model's successes. From launching weather balloons to running global weather-monitoring satellites, the scientists and systems at NOAA/NWS (and in this case, the UK counterparts) provide critical expertise and data.

I say this because it seems that earlier announcements where industrial deep neural nets "outperformed NOAA" likely encouraged the slash-and-burn Trump administration in its gutting of critical activities and centers of expertise at NOAA. The impression that industry can predict weather better than the government agencies totally misses that the industrial models utterly rely on government data for inputs. In fact, almost all weather reports you see---weather.com, TV, etc.---are just lightly repackaged products that NOAA provides for free on weather.gov (which you can access for free without ads).

Clench the other fist while writing. This loosens the writing hand. Something in the subconscious releases tension in the other hand. It's a trick I've used with reasonable success and it's immediately effective. It does require a bit of conscious effort, but overall it's a great hack.

What I think is really going on is an attempt to segment the market in favor of Google's strengths. It's a bet that models are "good enough" for many use cases even before they reach human-level intelligence, and Google is trying to capture workflows where quantity beats quality.

They are likely deliberately avoiding the SoTA race for a few reasons:

1. Their best models are marginally better than current SoTA releases. 2. They'd like to let Ant/OAI make mistakes with safeguards / let them get the regulatory heat. The unknown unknowns are huge with SoTA models (eg OAI accidentally hacking huggingface) and they are protecting their reputation. 3. They want to encourage companies to become cost conscious because they can likely win on price in the long run. Getting market share in "quantity beats quality" workflows forces companies to establish processes to choose the "cheapest acceptable model", which is a good environment for Google.

I've heard of capture technologies that literally pump organic matter down far enough that it sinks to the seabed (high enough pressure collapses air bubbles and results negative buoyancy). The problem is that your pumps need to operate using less carbon than you capture (easy with solar?) and that they need to be durable enough to pump billions of gallons of seawater with low maintenance (much harder, salt is terrible for machines).

It seems that we have two different stories here: in one, the new optimization theory represents a stark departure from the prior art, a sort of revolutionary new view of the understanding of optimization as applied to neural networks.

In the other story, the current understanding of optimization is a natural evolution of past work, where a new generation of researchers respond to social and technological changes, adapting and building on the work of the past, taking what's useful, downplaying the importance of some ideas, and inventing new language to describe concepts that seem most relevant to the current situation.

Both stories tell some of the truth. A revolution or evolution? Looking at the literature (eg the sibling comment here) shows that even today, convexity is used as an intuition pump for modern optimization techniques. But there are also new ideas that apply to the specific exigencies of neural nets, and downplayed ideas (eg convergence rates) that seem less relevant.

Yes, apologies, I didn't read the articles you linked before posting this. I did update the comment.

I don't think this changes the point, which is that most optimization methods used in AI owe a substantial intellectual debt to convex optimization theory.

I'm sorry this comment didn't sit well with you. My goal was to induce discussion by describing the claimed result (which was buried in the post), not to discourage it.

If you have more specific feedback on what you found distasteful, I'd be happy to hear it.

I'll point out that "does not work" is not the same as "not as efficient" :) But it does seem the Adam paper had an error.

I think that Nesterov's first order method is the most efficient general first order algorithm on convex problems, so anything else is in some sense worse. (Edit: removed incorrect ADAM comment.)

I'd push back on this. Most of the core optimization techniques (eg, ADAM, stochastic gradient descent) are straight out of the convex optimization literature. Generally you need to use optimizers that work well on convex objectives because near minimizers, functions tend to be convex. (Proof by contradiction: a non-convex point has a strict descent direction.)

The fact that neural networks are highly nonconvex has encouraged a lot of research, but it's more of the kind aimed at resolving tension: these methods are probably good for convex functions, why do they continue to work for nonconvex problems, and are there tweaks we can make to improve them in that setting? It's not a lot of de novo theory; more standing on the shoulders of giants, etc etc.

I know a bit about this field. This conjecture reads as somewhat more niche than the cyclic double cover conjecture recently proved by OpenAI, but nevertheless represents a real contribution.

You want to know how long it takes to solve an optimization problem, in this case over convex, lipschitz functions. (The restriction to a spherical domain is not really a restriction, you can just change variables for any bounded domain.) Anyway, showing upper bounds on time complexity is "easy" because it's just the runtime of your algorithm. Showing (nontrivial) lower bounds is usually much harder because it requires constraining all algorithms.

This proof apparently shows that the lower bound time complexity is equal to the time complexity of an existing 30-year old algorithm: it requires Omega(d^2) function evaluations to solve over this class of functions.

My gut says likely implies that d is the minimal number of evaluations if you have a gradient oracle because you can approximate a gradient with d function evaluations, but I'm not sure how hard it is to make that rigorous.

An example that came up in the article is medicine (or more precisely a medical patent), which also clearly has an extremely strong time value, particularly for lifesaving medicine.

My read was that MC = "market clearing price" not "marginal cost".

Being a bit more humble, perhaps the lesson is that the difference between the theory and reality highlights externalities that always exist in the real world that make the theoretical model miss a crucial piece of the real world. It's logically correct in some sense, but incomplete.

The tails could be heavy, it's true. This could make my usual 1-in-1000 heuristic overly conservative. Let's go with that, heavy tails. The data shows we are at something between a 1-in40 and 1-in-20 deviation. Normally (pun intended) that's ~2-sigma deviation. So... I guess it doesn't really seem like an epoch defining El Niño.

Fair, I had to click into the full size (I pinch zoomed on my phone). But now the fact that 1988/1989 are outliers raise a few more questions about the variance calculation. In particular, the title suggests that those weren't included in the variance calculation.

A 3.5-sigma event is a 1-in-1000 chance. Yet we have 2 of them in our data set that goes back only 40ish years? Something is suspect. Maybe I'm not accounting for the likelihood of a random walk excursion probability, but if a 3.5 sigma event isn't a 1-in-1000 event I have difficulty interpreting.