bananaflag

On the other hand, STEM will disappear last because, as long as machines won't be able to do something, the study of how to make them do it will be a part of STEM.

> I've seen people that do well in undergrad math degrees simply by massively memorising things and learning how to join them up to some level of degrees-of-separation, but seemingly completely fail to understand, for ezample, why even calculus is how it is. Because they are able to regurgitate the results and "produce proofs" this is never questioned.

If you want to catch them, surely you can find proofs they aren't able to produce.

> As someone who mostly only applies math, that strikes me as a peculiarly academic take.

Yeah I was talking strictly about preparing students to become pure mathematicians. No opinion here on other goals.

> Surely this implies these LLM generated proofs require the LLMs to have mathematical intuition . . . and honestly I don't think many people believe that

Yes, I believe that, it's part of what I was implying (I believe the LLM weights have some internal representation of math in the same way brains do that allow them to produce proofs)

A lot of existing research is already like that. AI will accelerate it.

> The person employing the socratic method must actually know the answer and where the student is in their mind.

The socratic method also has a much higher chance of revealing where the student is in their mind.

But the thing is, all have that icon+text style in the "tombola, cakes and drinks..." part. If I wanted to be distinctive, this is what I would have tried to get rid of at first, not the "visual style" (it's sort of like Windows XP visual styles, no matter how different they are the overall layout is still recognisably Windows).

As a math prof, I care(d) much more about proof than intuition, not because proof is more important, but exactly because intuition is (I'm a bit Chesterton-ish here haha). You cannot do proof without intuition hence, if you emphasize proof, intuition will take care of itself. Whereas if you emphasize intuition, students won't have any idea of what a rigorous proof should be.

Nice, it's pre-AI

3. He is not to be confused with Hermann Weyl.

I know people who did research on the side for a decade while having an ordinary job. They then returned to academia.

I couldn't do that, but I know there are people more passionate than me.

Well, it's not like humans are nigh-invincible demigods. We are quite fragile and could have killed ourselves as a species since way before AI. If AI wants to kill us, it shouldn't be that difficult. So I don't see why people are fixating on this question.

> I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.

What stops you? I can fathom it alright.

I am always suspicious of these arguments like "a machine won't do thing x which humans can". These always basically pre-suppose that humans aren't machines. If that's what you're arguing for, say it outright.

> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.

Glad he's at least admitting it as a possibility.

Also: you know, you don't need to rely on lossy proxies like how much or how little sex do the arguments' proponents have. You can just read the arguments!

Thing is, you can say that about any technology (not being here necessarily pro AI in math, but I think we need better arguments).

I admit that I have always looked at an explanation like yours with x1,...,xn when using fold because I could never keep it straight in my mind.

A professor of mine had an anecdote of meeting Serre and complaining to him about Bourbaki style and how hard it is for students.

Serre's reply was "But we never wrote those books for students! We wrote them for researchers to have a handy reference for all proofs of basic results."

Funny how it can do everything but not chat. Sort of how when I was a kid I thought of a medicine that could cure any disease except the common cold.

They confuse it with public domain, which means exactly that.

I first wanted to say fire, even though it's a cliche, but then I thought that in antiquity we used like everything without anything that would qualify today as understanding. Also now we have a lot of stuff that we "know" it works based on complicated numerical simulation.

I think the most "understanding" we ever had was in the 40s-50s designing nuclear bombs with slide rules. It was the culture that produced the idea of psychohistory.

> Medicine is still the wild west.

Reminder that we have no idea how anesthesia works.

Yeah it s because they have a strong prior on unsolved problems being unsolvable.

Once the idea of AI routinely solving conjectures enters the training data this encouragement will disappear like 2023-era prompt engineering did.

The way the platform works can be shit while the content on it can be good.

It won't be necessary in a year when the information "AI is superhuman" in all its guises enters the training data.

Toasters don't toast toast, toast toast toast.

I know, yes. I was talking about the current Bostrom/Yudkowsky iteration.

Re Butler, you may enjoy this Chesterton pastiche if you haven't already:

https://slatestarcodex.com/2017/04/01/g-k-chesterton-on-ai-r...

> which you clearly haven't done

I have read all that, I follow e.g. Zvi Mowshowitz's blog.

> You're trusting the leaders of for-profit companies to do right regardless of their skewed incentives

I'm not trusting them at all. I have seen for years how they don't do anything about alignment. OpenAI has disbanded countless alignment teams. But if their skewed incentives are for once bringing them into the right direction, I'm not gonna stand against them.

> OpenAI said GPT-2 was too powerful to release

Because they thought such a text-generating machine will bring about a golden age of fake news. Seven years later I have relatives who send me every day some long-winded AI-generated bullshit. And I have seen with my own eyes how this sort of thing influences elections.

> Antropic said Mythos was too powerful to release. Both are GA now

Mythos was too powerful to release, hence it is not GA now. They released Fable, which is Mythos plus guardrails.

To everyone who believes that this is some nefarious plot by CEOs to bring about regulation or to hide that they've hit a wall or to make their models seem more capable or whatever:

The AI risk arguments are philosophical arguments that are 20-30 years old and don't have anything to do with the current corporate landscape. They were thought of by people who really care about the issue and all the "gotcha" lines have well-developed counters. Of course they might be stuck in their ideology but they deserve a bit more credit.

I don't doubt that AI CEOs can profit from a crisis and that they don't necessarily have our best interests at heart. But this is the same way governments during COVID tested how much they can restrain freedom. This doesn't mean COVID doesn't exist or that vaccines are evil or ineffective or whatever. Same way here, if there is an advantage to CEOs to regulate things doesn't mean AIs can't kill us all and that regulation might bring us more time to solve alignment.

Also: I accept I might be wrong and AI risk might be a fake issue and/or alignment might be impossible, but please bring better arguments if you want to convince me.

You know how that story ends right?