bananaflag

If AI is so good at research then maybe instead of people researching comparatively easy problems which only get you a publication, they will research problems that matter. (I'm not dismissing here problems just because they are "easy" or "not practical", I am sure that there are a ton of worthwile ones among those, I am dismissing just those which are artificial and only researched out of publication incentives.)

I guess a lot of highly intelligent people relied on their smarts to coast to tenure and then all the way to retirement.

Yeah, Zvi calls it "the forbidden technique"

I wonder how it will be at solving open math problems.

Link for curious people

https://en.wikipedia.org/wiki/Illegal_number

Note that they're saying "proprietary models". Probably open models will reach this level in a year or so and then this discussion will be moot.

Before 2024, I once went to an ATM to retrieve money and selected 50. Note that I selected it from a menu, not typed it. The ATM then told me that it cannot give me 50 because it is not a multiple of 5.

Did you complete it?

You mean "Artificial Intelligence meets Natural Stupidity" by Drew McDermott

https://dl.acm.org/doi/pdf/10.1145/1045339.1045340

In principle they could output meaningful such language if they were capable of metacognition, which so far doesn't seem to be a goal of AI developers (and rightfully so, since they achieved so many miracles bypassing it).

> The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.

Can't you just not grade it at all? If the student has done their homework, just consider it passed. And then don't count it at all for the final grade.

There's a book about that, here is its ACX review:

https://www.astralcodexten.com/p/book-review-deep-utopia

Yeah because people don't like to read. They do it if they're forced to but no more.

Is it consuming too little science fiction that has people thinking like this?

How would you propose to build an intelligent being which would be incapable of doing this sort of thing?

I mean, ever since AI was a research program, it was assumed it will some day achieve its goal -- in particular, make computers be creative.

Shouldnt this be "misrepresents"?

> Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.

If someone discovered how to summon demons to aid them in robbing banks the main issue wouldn't be "but who has the responsibility for the crime, the human or the demon", it would be "OMG DEMONS".

Seriously, I don't get what sort of world these people are living in.

One of the authors is one of the stochastic parrot authors.

> As a Nobel Prize winner, I am surprised that you have not noticed how Tao and Gowers were steering everyone in the AI direction for years

I did notice, this is why I had thought that they had thought through the consequences. But, unfortunately, it is likely they imagined that AI would plateau somewhere around the "smart undergraduate" level and that they will stay relevant in pure problem solving.

> I don’t have a solution here, and I’m worried it’s coming to all human fields. So it’s more of an ability to observe what might happen to all of us.

You can also look (as I did back in 2018 when contemplating the end of math) at how chess or Go players adapted mentally to the idea that whatever they may do, a machine could do it better (Lee Sedol famously quit Go).

https://archive.is/m9BYI

> I knew mathematicians were a smart bunch, but honestly, the events of the past few weeks have really given me a new level of respect for them.

As a mathematician, I am a bit disappointed by my (admittedly illustrious) colleagues.

I get the need to take it slowly (and I am a quite impatient person, so I shouldn't get to decide stuff like this), but everything said feels a bit too sour grapes for my taste.

Ok, maybe AI did not solve the field (I believe it will, btw), maybe there is a need for human "understanding", but:

1) They don't seem to consider even the possibility (not the certainty) that they might be wrong, that math as we know it is gone, and we cannot "adapt"

2) They seem to have been oblivious all these years about AI eventually reaching this point (at least I personally wasn't, I predicted this stage back in 2018)

I frequently say what you say, but there is a more charitable reading: today's AIs are not "minds", as they are not stateful. Maybe continual learning ones (like the mini AGI exhibited today on the HN front page) will be perceived as "soulful".

I think to me LLMs had the effect of noticing much more the author, the intention behind human-made works of art (books, movies etc.). Before LLMs, I used to frequently consume media in a way as it were generated by a mindless process. Now it's like everything which is not AI-generated has more meaning than ever before, a bit like hypomania.

This is the first thing I see in my life that really looks like proto-AGI, it deserves its name.

Okay, fair enough.

Okay here's one of his explicit predictions, from 2018 (before LLMs) about 2023:

"If AI can generate images and even stories to a prompt, everyone will agree this is totally different from real art or storytelling."

https://slatestarcodex.com/2018/02/15/five-more-years/

Scott Alexander predicted LLMs will write math proofs in 2019:

https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g...

Of course people were skeptical:

https://www.reddit.com/r/slatestarcodex/comments/aslze7/gpt2...

Now I want GPT-4.5

I don't think research communities should be destroyed.

Best case, they still matter.

Worst case, AI kills us all and it's irrelevant that we "wasted" money on research.

> Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done.

In the (extremely) short run, yes. In the long run, those jobs will also be done by AI.

It's like chimpanzees seeing human society and saying "look how complex it is, imagine how many chimpanzees it needs to maintain it".