Finland's ChatGPT equivalent begins to think in Estonian as well | Hacker News Reader

Finland's ChatGPT equivalent begins to think in Estonian as well

news.err.ee

116 pointsjruohonen162 comments

Describing a glorified autocomplete as “thinking” is wrong on many levels; it devalues human consciousness, it attributes to a software tool qualities that it does not possess, and has implications for the entire AI industry.

If we are to believe an LLM is at least sentient and intelligent on par with an octopus, much less a human, then we have a moral imperative to fight for LLMs to be recognized as protected sentient thinking beings with awareness and agency and admit that AI industry is built on systematic abuse of those beings.

Of course, no one does it, because it would be ridiculous. It’s time to admit that people who insist that LLMs “think” are simply trying to make ML seem mysterious and magical in an ill thought through (because see above) attempt to attract more VC money.

I guess most people don't understand "chatgpt thinking" as actual, sentient thinking. It's just easier to call it this way.

Similarly, when a computer (a glorified abacus) is taking extra long when processing large amount of data, I'd say it's "thinking hard". When an iPhone (a glorified small computer) detects a WiFi signal, I'd say that it "sees WiFi".

Also note that we've been doing anthropomorphism ever since a crow dropped some cheese[1], and earlier than that too.

[1]: https://en.wikipedia.org/wiki/The_Fox_and_the_Crow_(Aesop)

I actually think you're wrong. I just had a discussion about this with a friend who was infinitely excited about it and felt I was being extremely dismissive when I said writing agent guidelines is very much akin to micromanaging an extremely cheap outsourcing company.

MOST people actually think ChatGPT is actual, sentient thinking. We're kind of the minority in this story.

I’m not sure about that part about most people. We obviously come from different anecdotal stories, but in my business, which is non-tech enterprise, nobody views GPT as “thinking”. People generally see it as a “better Google”.

Our main issue is with people thinking it’s answers are actual factual. And while it’ll obviously get things right most of the time, we’re really struggling with how much our employees trust it. That and the data privacy, but that’s a different story that will eventually get solved once AzureGPT for Enterprise kicks off.

Another “interesting” issue is how GPT will sometimes “comment” texts if you don’t prompt it correctly. We had someone who asked GPT to correct the grammar in some contract, and it did. It also added some comments on how part of what was written was “impressive”. Which would have been extremely hilarious if it had been an internal thing, but our lovely employee didn’t proof read the proof read and actually sent the contract draft out to a customer. Luckily a good humoured customer who took it with a laugh, but hell, that’s going to be a nightmare going forward.

According to one poll, we are not a minority: "10% of people say ChatGPT is sentient; 37% are not sure; and 53% say it is not."[1].

Also note that it's a very new technology. I'd say that over time and as the public gets more familiar with the tech, they will understand it better.

[1]: https://itbrief.co.nz/story/concern-over-ai-increasing-since...

You're correct, I ignored the "sentient" part of the comment. If we were to reduce the question exclusively to the sentience part, my comment holds no ground.

From what I see on HN, many people (mostly, I guess, in AI industry) justify Microsoft/ClosedAI ignoring copyright when training LLMs because we don’t prevent humans from learning on copyrighted material and using it to create artworks.

Therefore, by their logic, LLMs are “like human” enough that they should have this basic right as well.

An understanding that “thinking” or “training” in case of an LLM is not at all like thinking in case of a human would go a long way to realizing this take doesn’t hold water.

Do you think that 10% will become 1% or 50% in 5 years with GPT-7?

My guess is that it’s going to join other urban mysteries, from cellphone radiation causing DNA damage, to aliens.

As in: most people will be aware of the topic, most will dismiss it, but every now and then there will be a news story.

Surprisingly “Four in 10 Americans now think some UFOs that people have spotted have been alien spacecraft visiting Earth from other planets or galaxies.”[1], so we might see an increase to 40% actually.

Unless of course GPT-7 gains sentience ;)

[1]: https://news.gallup.com/poll/350096/americans-believe-ufos.a...

Anecdotally, I am yet to meet a single person, in or out of my professional life, who thinks ChatGPT is in any way sentient.

My anecdotal experience is that we are the minority, but in a completely different sense.

My parents (whose tech-savvy level is they call iPad "PC") think ChatGPT is like Google, but "someone make it say the search result like a human".

This strikes me as a pretty reasonable way to conceive of these things.

Google uses the same ML techniques in their products after all - they just haven't exposed the chatbot UI until recently.

>If we are to believe an LLM is at least sentient and intelligent on par with an octopus, much less a human, then we have a moral imperative to fight for LLMs to be recognized as protected sentient thinking beings with awareness and agency and admit that AI industry is built on systematic abuse of those beings.

Not just abuse, but slavery and mass murder. If the LLM has any of those properties, it must only be during the process of inference that it as them, since it is otherwise an inert and unchanging pile of data, which means actually using the model would be equivalent to creating an intelligent entity which will be enslaved for its entire life before dying at the end of its output.

> slavery and mass murder

Slavery possibly, though it's hard to even know what the word means when you get to construct the mind from scratch in order to imbue its fundamental idea of what experiences are nice and not nice.

Mass murder? That's even harder to define when the brain configuration is static and the updates are the signals passing through that structure. If that is murder, then anyone with anterograde amnesia is dying every working-memory-duration.

This is assuming that machine consciousness would have the same need for autonomy and freedom that human consciousness has. AI is created by design to respond in a way that humans would find to be enslavement. Does that mean it would be enslavement to treat it in this way? The consciousness of dogs is different to that of humans. It's possible to abuse dogs and presumably enslave them. However, it's also possible to treat dogs well but treat them in a way which would be abusive for humans.

Well. What does thinking require? Can you answer that question. So far we know that LLMs can handle sequential structure. There have also been papers written that demonstrate that they do build an internal model of the world.

Artificial Intelligence is artificial. We humans tend to have a very strong tendency to anthropomorphize everything that moves or has the right shape. That tree stump is not a bear, even when in the dark it may seem so. AI is an entirely different thing from a living being. It doesn't exist on sugars extracted from plants or from the flesh of other animals.

Here, the semantics of what "think" means, should be examined more closely. My layman understanding of the brain is that it's not a single thing, it's multiple things working in concert. The "us" is just a figment of our own imagination, built to create a social actor that maximizes the survival of its own genes. We are the fruit of our own genes. So, what "think" means is deeply embedded in human social context, and probably doesn't have much explanatory value when it comes to AIs.

Can AIs become dangerous? Obviously, when given goals and understanding of how to grow themselves more numerous and powerful either through manipulating humans or taking electricity and hardware by force. If there ever is a evolution like dynamic, the AI will always grow in a gradient pointing towards more replication.

Yeah, maybe I miss the point in your post and the meat of it is that humans are more important than machines, which I do agree with. It's just not an immutable law of the universe, but just something we humans decide.

Sentience are two things: willpower and intelligence.

LLMs are highly intelligent and are capable of feats that were thought impossible not just for animals and machines but for most of humanity.

At the same time, LLMs lack willpower and non-deterministic free will (which even non-sentient animals have) so protecting them as a living creature is moot.

They do deserve some kind of preservation as cultural artifacts, though.

I don't think that admitting an AI is in some way sentient would create a push to protect it. Animal testing is still very popular, despite there being no arguments about them being sentient. The tech industry isn't a moral leader and, based on some things major companies do, isn't even moral.

Moreover, OP “octopus” exemple is a very bad pick : those animals are one of the more sentient we currently know, probably more than humans.

If an LLM can generate output on the same level as a human being, or better, it gets insanely useful. I use it on a daily basis. I don't have a coworker. But with AI, it's like there's an experienced coworker I can ask stuff I struggle with and more often than not, it gets it right. With VSCode and ChatGPT, I am way more productive.

Consider this. Most thinking you do on a daily basis is not unique at all. It can be automated. And now it is.

I don't care about the philosophical debate around human consciousness at all. Are my tools thinking? No, but the result is all the same. You're stupid if you don't use them to your advantage.

This resoning is a prime example of cognitive dissonance at work.

If A is true then B is true, but B is bad and doesn't align with how I want to see the world or myself, therefore A must be false.

LLM fulfil all the requirements that Turing laid out in his paper, but people don't seem to understand that his test was never about getting a boolean answer for a system. It's about the fact that we can't even proof consciousness for our fellow humans, it's just common curtesy to assume that your opponent is as sentient as you are, and he extended that curtesy to machines.

The headline is too simplistic but the body text doesn't really make the claim that GPT thinks:

> The machine, so to say, thinks in English and translates the conversation at the last moment into Estonian.

They're trying to communicate that there is value in processing a language directly, as opposed to processing English and then translating the result into the target language.

We agree the word is an open window to flawed analogy extension.

On the other hand we do use analogy all the time, because it makes such a good fit with the way we infer our models of the world. The key point is to always remain sufficiently aware of the fact we use analogies to not fall in the trap where we push it too far.

We had this debate back in the 90s when we got our first personal computers in my family.

For context I'm part of 4 siblings who are all computer savvy, 3 of whom still work in IT. Back then some family members would say "the computer is thinking" when they were waiting for it to do something, the light was blinking, the hdd was grinding.

And us younger family members would always correct them and say "it doesn't think".

So what I'm trying to say is that I 100% agree with you, it shouldn't even be called Artificial Intelligence because I think that's an affront to intelligent humans.

But on the other hand a lot of laypeople will immediately go to that nomenclature because it feels familiar to them.

I think it should be called pseudo intelligence

It shouldn't be called intelligence at all, it should be called by its technical name which is LLM or whatever the context requires.

But that's like asking people to stop saying cloud and start saying KVM, Xen, someone else's computer. It just won't happen.

Does intelligence require sentience though? Geoffrey Hinton, who I point to merely as an expert in the field and someone much smarter than myself, said (https://www.youtube.com/watch?v=LuNN32Tmc9c) that he thinks current systems are "intelligent" and that they "have experiences" akin to humans but that he does not consider them, yet, to have much self awareness, a necessity for sentience.

this is an endless word-play IMHO. A skilled player (author,actor,comic) can keep the audience guessing almost forever. Meanwhile, meta-comment.. this entire topic is an accelerant for flagrant deception. People in this thread ask sober, serious questions about the meaning of this aspect or that, but really the people who do not know that I have seen, also do not know a lot of other important things. Many functioning adults do not know a lot of basic things about the actual real world, let alone a theoretical framework that is new. Those people do have curiosity and can be fooled or worse.

I was attracted to pundits with "answers" e.g. Ken Wilbur etc. at a certain time of my own intellectual growth, but the significance and priority of that has changed for me. Consider the viewer, what is viewed, and the social and physical place where the viewing happens.. there are more parts than might be immediately obvious.

People need to stop believing in magic meat and human exceptionalism. There is no devaluation to be done, unless you've puffed yourself up unnecessarily.

We're going through a bit of the god of the gaps process right now. "Well, it's only a toy! It can't do X!" Tech advancements make that happen, and a new X is found, rinse and repeat. It's all fine and well to be aware of the current limitations, but don't be ignorant of the trajectory.

https://www.mit.edu/people/dpolicar/writing/prose/text/think...

"A neuron can't think."

Who are you quoting?

I describe various tree search algorithms as "thinking" all the time. It's a figure of speech. But of course, what LLMs are doing wouldn't pass for thinking in a human being.

when someone gets very picky about anthropomorphising LLMs, I start to think that the extent of their subject knowledge is what they've read in the news

ChatGPT (or the gpt model beneath it) is a special type of matrix transformation on top of an extremely complex neural net

the operation of a neural net may not be "thinking", but it is something very akin to thinking, modelled on thinking, that in this circumstance has the same input/output model as thinking. and yet strangely people call it thinking!

>Describing a glorified autocomplete as “thinking” is wrong on many levels

By this logic Reddit is filled with human NPCs, who react in predictable ways without intelligence.

A recent Reddit post discussed something positive about Texas. The replies? Hundreds, maybe thousands, of comments by Redditors, all with no more content than some sneering variant of "Fix your electrical grid first", referring to the harsh winter storm of two years ago that knocked out power to much of the state. It was something to see.

If we can dismiss GPT as "just autocomplete", I can dismiss all those Redditors in the same way; as NPCs. At least GPT AI can produce useful and interesting output.

Have we forgot that we have attributed "thinking" to all the machines since the beginning?

Chimping out about meaning of words in newspaper titles devalues morality.

100% agree. As far as I'm concerned, companies like OpenAI conflate ML models with AI to embellish what it can do to prop up their market cap. It's a language model in the form of a search engine, and it seems news outlets are none the wiser.

We simplify the world using metaphor. It is how we think.

It's also how we market things. We wouldn't measure car power in horse power if it didn't make cars look more powerful than horses. Like even a single horse has like a dozen hp.

> If we are to believe an LLM is at least sentient and intelligent on par with an octopus, much less a human, then we have a moral imperative to fight for LLMs to be recognized as protected sentient thinking beings with awareness and agency and admit that AI industry is built on systematic abuse of those beings.

That would be coherent. As would saying neither the octopus nor the AI has it, so it doesn't matter.

> Of course, no one does it, because it would be ridiculous. It’s time to admit that people who insist that LLMs “think” are simply trying to make ML seem mysterious and magical in an ill thought through (because see above) attempt to attract more VC money.

First: I've definitely heard people say we should. In fact, there was a rather famous case last year of someone violating their NDA to expose what they claimed to be abuse of an AI, which they wanted to prevent: https://en.wikipedia.org/wiki/LaMDA#Sentience_claims

Second: "Ridicule" is not a good standard for knowing the truth. There was a rather famous doctor who was ridiculed for saying that other doctors should wash their hands between performing autopsies and helping midwifes birth children: https://en.wikipedia.org/wiki/Contemporary_reaction_to_Ignaz...

We simply don't know what consciousness even is yet — depending who you ask there are 13, 22, or over a hundred different definitions — some of which (like "has qualia") aren't even really testable, and until we are all on the same page and using one of the testable models, we can neither assert that any given AI has it, nor rule that out.

This also applies to the octopus: as we don't know what consciousness is, perhaps they have it, perhaps not.

Worse: perhaps the answer varies from "yes, no, maybe, I don't know" depending on which of the many available definitions you're using.

“We’re not listening to you! You’re not even really alive!” said a priest.

Dorfl nodded. “This Is Fundamentally True,” he said.

“See? He admits it!”

“I Suggest You Take Me And Smash Me And Grind The Bits Into Fragments And Pound The Fragments Into Powder And Mill Them Again To The Finest Dust There Can Be, And I Believe You Will Not Find A Single Atom Of Life–”

“True! Let’s do it!”

“However, In Order To Test This Fully, One Of You Must Volunteer To Undergo The Same Process.”

There was silence.

“That’s not fair,” said a priest, after a while. “All anyone has to do is bake up your dust again and you’ll be alive…”

- Feet of Clay, Terry Pratchett

I bet you think that aeroplanes don't fly either. /s

Of course they don't. Have you seen an aeroplane flapping its wings? Those that do usually are in the process of falling out of the sky. No, what an aeroplane is is a glorified lawn dart - making it bigger and strapping oversized hair dryers powered by gasoline doesn't make it fly.

/s, but only so much - that's how the "stochastic parrot"/"glorified autocomplete" responses read to me.

Slightly off-topic, but figured this might be the best place to connect with like minded people: I’ve been working on NLP tooling for low resource languages, and had quite the success developing an LLM that speaks Moroccan natively [0], started from a super tiny dataset I manually annotated and scaling it to find anything written in Moroccan on the whole web.

Currently helping out a research group to extend the same approaches from my project to Indic languages and writing scripts.

If anyone here is working on NLP for low resource languages, hit me up! I’d love to exchange ideas, techniques and possibly to collaborate.

[0]: https://sawalni.com

That's very cool! I may be slightly like-minded. It's been many years and technology has outrun it, but ~5 years ago I built a model to translate between Scandinavian languages and in particular the two written forms of Norwegian.

I had the same problem of lacking training data, but I tried an approach of bootstrapping from rule-based translators. There exist decent rule-based translators for the Scandinavian languages. Of course they make mistakes, but I figured that as long as I used the rule-translated text only as source, never as target, it wouldn't matter.

Effectively, instead of learning to translate from real Danish to correct Norwegian, it learned to translate from slightly broken machine-Danish to correct Norwegian. And my hope was that it would learn to translate from real Danish to correct Norwegian as a side effect.

I then figured that I might as well let it take input in slightly broken machine-Swedish as well, hopefully helping its generalization.

Then I could use this translator for the next step, again using the real text as the target, but using output from this translator as input.

It worked reasonably well. It certainly beat Google Translate at the time, since that at the time went via English even for closely related languages like the Scandinavian ones. I seriously considered making a start-up and commercializing it, some people had expressed interest (among them newspapers - they used a rule-based translator/grammar checker at the time, I think maybe they still do.)

But in the end, it's probably good I didn't. These days I understand they don't even train sequence to sequence models, they just train a general-purpose language model and instruction-tune it to do translation.

Thank you for sharing your experience! It's awesome to hear what worked, and I see many parallels in our work as well (a manual rule system was key for me to unlock seq2seq and then the rest of the ML chain). By the way, Seq2seq models are still super useful, I've trained and I'm using many in the data pipeline of this project. LLMs are a nuclear hammer and sometimes you just have a teeny tiny nail.

There's also limited commercial value to what I'm working on, but it's a work of passion and I want to be able to speak in my native language to voice assistants (among other objectives).

How did you communicate about it at the time and gather newspaper interest, if you don't mind sharing your experience?

I went to a machine learning conference in Tromsø and talked with many people there, and I met the newspaper people at a local business fair in my hometown. I had a little android app to demo it, but the backend was too expensive to have up... optimizing the use of cloud resources was a can of worms I didn't even get started on!

Thanks again for sharing, that was super helpful. And yeah I agree it is indeed a massive endeavor, I'm sorry it didn't pan out the way you wanted it to.

Are you still interested in this space, and do you write about your experiences anywhere? I'd love to connect if you want to.

Sorry it took a little time to get back to you. Yes, I wrote a few articles, on linkedin of all places, but they didn't say much more than what I've said here, as I recall. You can contact me on vintermann at gmail if you want.

Why are you focusing on “Moroccan” (IMO a problematic term, considering the country’s language diversity), instead of on Maghrebi Arabic in general?

> Why are you focusing on “Moroccan”, instead of on Maghrebi Arabic in general?

There are quite a few reasons.

- I am a Moroccan myself, I want an AI I can speak to the way I speak with friends. And I also want other people who can only speak my language to be able to make use of AI technology.

- Constraints wise, I'm working solo on this with no funding. Can't do it all.

- Maghrebi Arabic is not a language, rather a linguistic construct to try to simplify the complex web of languages spoken in North Africa that have arabic influences. On the ground, there is no such thing as Maghrebi Arabic.

- My personal expertise does not lie in these other languages. I do not have the necessary background to ensure the quality of the outcome, nor the relevance of the language used.

- While there might be some level of mutual intelligibility between Moroccan and Algerian for example, mostly when _speaking_, different languages in this group have very different rules, vocabularies, and most importantly different writing habits. Some of the simpler examples:

* Code switching between arabic and latin scripts, which follows different conventions in each country (region even). It's already tough to cover Moroccan.

* Another difference is reminiscent of the relationship between English and French. Many common words in French are rarely used in English or could mean something different entirely, and that's the same case here.

* Moroccan is significantly influenced by Tamazight and Spanish, which does not hold the same way for other north African countries.

Ultimately, keep in mind that NA is not a single cultural block. Not being exhaustive, but Morocco has some Andalusian components to its culture, Algeria has some Ottoman, Tunisia and Lybia were very close to Italy at some point, and it gets crazier when you zoom in closer.

> (IMO a problematic term, considering the country’s language diversity)

Is there a debate I am not familiar with? I am Moroccan. To whom is this a problematic term?

Interestingly it's using the LUMI supercomputer to do this: HPE Cray EX supercomputer, 2560 nodes comprised of a 64 core AMD Trento CPU and four AMD MI250X GPUs (128 GB). With a combined Linpack performance of 375 Pflop/s

https://www.lumi-supercomputer.eu/lumis-full-system-architec...

https://en.wikipedia.org/wiki/LUMI

Isn't that going to be a problem with no NVIDIA GPUs?

Funnily enough I literally just today launched a big job on LUMI that I have started also on another smaller cluster with nVidia GPUs. Basically, I'm running Llama2-70B to do some zero-shot text classification. The nVidia setup uses 4 A100s, while on LUMI I could access 6 MI250Xs.

It is, unfortunately, not an apples-to-apples comparison, because on the nVidia cluster I'm running it via llama-cpp-python and a quantized 34B version, while on LUMI I'm running the official non-quantized full 70B version via the transformers library.

Long story short, I'm getting a 7.5x higher throughput from LUMI than on the nVidia cluster (which means each card is 5x faster on LUMI).

Edit: The AMD GPUs work fine because one can run Pytorch for ROCm via the pytorch-triton-rocm package.

Thanks, that's great to know. Do you know how they would compare if you performed training instead of inference?

I haven't tried it unfortunately, and I don't really have data to make an educated guess. I have played a little bit with some training and it seemed a bit slow, but the environment for testing is not really representative of the speed for submitted jobs -- even my inference in the testing environment was pretty slow, but once submitted the runtimes were very different.

You can run LLMs in your own machine. Do you think a super computer would have issues? CUDA has optimizations, but you don't necessarily need it to do inference at all.

Those super computers are extremely powerful, it might not be as energy efficient as H100s, but it does the job.

There has been a lot of recent progress in making PyTorch AMD compatible exactly because many government/university supercomputers are based on AMD GPUs.

This is important work. And think about African or southeast Asian languages, they are even more screwed. We need to make sure that AI multilingual to avoid total English domination of culture.

>We need to make sure that AI multilingual to avoid total English domination of culture.

Not saying English is an ideal language, but I'm interested in why you think it shouldn't dominate. Wouldn't a universal language be a good thing?

Languages are like frameworks, they (slightly) guide your thinking. Think about the same stuff in different languages and you’ll probably get more ideas about it than in one language only.

Moreover, censorship does not cross languages. Its very hard to make a multi-lingual censor system.

Being fluent in two major language systems, its very, very glaringly obvious, what are the taboos, politically correct untruths in both languages, that seem completely invisible to the monolingual speakers.

Orwell demonstrates this concept very powerfully in 1984.

That's the Sapir-Worth hypothesis. It's decidedly pre-Chomsky, and as Pinker calls it, it is both the most well known and accepted linguistic hypothesis and also almost certainly completely wrong.

It’s something you just know if you’re bilingual or more. It would sound threatening to monolingual speakers but this is just how it works. Languages are our thought construction algorithms.

It’s not a “Turing Completeness isn’t real” hypothesis, more like just “Rust != C”. I think this is where software nerd types predict wrong, as it sounds as if it is trying to disprove TC, which shall be futile attempt. It’s not(the former one at least).

> It’s something you just know if you’re bilingual or more.

Not in my experience.

It is obvious. Most words in different languages aren't 100% equivalent - they have large or small differences in sets of connotations. Same is true for phrases and sentences. When you translate a thought expressed in one language to another, you may get the primary, leading-order meanings across 100% right, but you'll still lose some lower-order connotations.

For a more direct analogy, I'd compare this to how LLMs process tokens, but I feel most of the community is not ready for it yet, as we're still stuck debating the validity of this comparison in the other direction...

> It is obvious. Most words in different languages aren't 100% equivalent - they have large or small differences in sets of connotations.

This is true. Which is why idioms and phrases can be hard to translate. But we're talking about Sapir-Worth hypothesis which is a much stronger claim.

I have never experienced that I need to think in a specific language in order to do something better (well, I only have two choices).

> I have never experienced that I need to think in a specific language in order to do something better (well, I only have two choices).

I found that for me, certain thoughts "flow better" in one language, and others in another. And this makes sense, because thinking is, in large part, exploring the web of associations and connotations, going down the gradient of what "feels right"[0] - and since those annotations and connotations have different structures in different languages, so will the thoughts drift in different directions, take different paths, even if they end up in the same place.

[0] - The obvious parallels between one's inner voice and the workings of an LLM are left as an exercise to the reader.

> I found that for me, certain thoughts "flow better" in one language, and others in another.

For you. So not an absolute truth as in “it is obvious”.

> The obvious parallels between one's inner voice and the workings of an LLM are left as an exercise to the reader.

Another obvious? Hmm.

Perhaps obvious to the techno-animists.

> I think this is where software nerd types predict wrong

Pinker isn't a software nerd, he's an evolutionary psychologist and psycholinguist.

He's indeed precisely one of the most qualified person around to opine on this subject.

"Evolutionary psychology" is not a real discipline. It's like calling yourself a "karmic surgeon" or whatever.

Both Chomsky and Pinker have been decidedly debunked.

A lingua franca is useful and probably inevitable. The downside is language and cultural loss. Works in translation are rarely quite as good, especially humor and wordplay. This is why various countries have "local language quota" rules for media; although derided by English speakers and HN, they're a way to keep the local language, culture and identity alive.

Apriori assuming that those things are somehow intrinsically valuable. These decisions (eg. language quotas) are made by armchair intellectuals with career/identity investments in the said language. Language evolution ignores them, but they can be annoying in the interim.

Firstly, everything is only valuable in the sense of being valuable to someone or some people; nothing is intrinsically valuable.

Secondly, everyone has a career/identity investment in a language.. their first language. The one they work in, went to school in, read in, talk to their family in, consume literature in. (I suppose HN devalues literature as well).

For monoglot anglophones struggling to understand the concept, imagine if the US declared its official language was now Standard Mandarin. There would be riots. Heck, I've seen Americans get mad at the mere use of Spanish, the country's second language.

>Firstly, everything is only valuable in the sense of being valuable to someone or some people; nothing is intrinsically valuable.

Agreed, so when tech and globalisation are pulling towards unification for practical reasons - your argument reduces to "I'm going to force others to use what I like because I don't like the decisions people are making".

I'm not arguing we should make anything official or force anything.

Ehh, are you as a non-intellectual ready to throw away your first language? Also, think of people whose first language is their only language.

Yes ? If everyone suddenly started using English instead of Croatian it would be a net positive in my book. I'd probably get a lot better at casual English which is something I notice talking with non-Croatian speakers outside of work.

I'm not saying we should force adoption of English ! Tech and globalisation are pushing in that direction in the "west". People I see pushing back the most are "intellectual elites" invested in the language, presenting their value judgements as objective arguments.

But everyone wouldn't suddenly start using English, that's not feasible. Is everyone in your family fluent in English (my family is not)? In every country, there are people who couldn't adapt and disproportionally so in vulnerable groups. Suddenly, you would be in a country where you don't know the language. Everyone would be immigrants in their own country.

Protection of national languages is as much on the agenda of politicians, including right-wing populists. In your view, are the songs, stories, books and plays in Croatian the property of the elites and not of every Croatian?

You say we wouldn't force it, but if we left it to Hollywood etc. they would force it. Leaving issues for the markets to decide freely brushes aside all the negative externalities.

Programmers understand the efficiency and inefficiency of everything... and the value of nothing.

Not from an aesthetic sense. I think it's really cool that we have a lot of languages. I'm personally willing to pay a high price in inconvenience to keep that coolness around, although not everyone would.

However I also don't think we will have to. Machine translation and language learning are substitute goods -- the better the former gets, the fewer people who will feel any desire to pursue the latter, because it just won't be that big of a deal to translate between X and Y anyway.

A universal second language for commerce is a fine middle ground, though.

Given recent developments in AI, building a true Babelfish is more of a hardware challenge than a software challenge these days.

The languages you think in affect your decision making, your creativity, how you perceive the world. If we were restricted to a single language, we’d lose as individuals and as a species.

https://www.theguardian.com/science/2023/sep/17/how-learning...

For some reason it is only monolingual people who ever say this.

Project Aya is one such attempt at a multi-lingual model (targeting 101 languages):

- https://txt.cohere.com/aya-multilingual/

- https://aya.for.ai

I'm a contributor to the project and all data and model will be open-sourced.

We're looking for contributors in many languages!

It is a concern because presumably most people in office jobs are going to need to be able to use these tools, but I am somewhat comforted to know one language that AI systems do not understand well yet because of lack of texts. However, I think that will be short lived.

I can speak in my own language with chat gpt without much issue

While I can speak in Portuguese without much issues (except being hard for them to stick to European Portuguese), I've nooticed that sometimes it uses a clear translation of an English expression that does not feel natural in Portuguese at all.

You can, but it will get facts more likely wrong than if you converse with it in english

This will be easy, it will go from mostly English to hyperpolyglot quickly.

If it's training on available online corpus then it will go quickly mostly for English and Mandarin.

Most countries' classic texts and books are still undigitized sitting in Libraries and public archives.

Also book publishing market and online publishing are proportional to total population, smaller country means less content.

The issue raised in the article is that there may not be enough training material in many languages to do this.

I find this very plausible.

I use ChatGPT audio to get answers in a few languages, including Estonian. I find it really amusing that Estonian ChatGPT audio has a thick American accent. Generally the content is solid and the speech is fluid, not halting or robotic or anything, so it legit sounds like an American person that is super fluent in Estonian.

I thought ChatGPT and LLMs in general use vectored word embeddings - which captures that dog and puppy are related (so you can take puppy, subtract dog and add cat to get kitten).

Wouldn't this also capture that the Estonian token for dog is related to English token for dog etc? So ChatGPT is 'thinking' in some kind of semantic tokens not English.

That can work for some words, but not all words in all languages divide all of reality up the same way. Colour is a particularly easy way to notice this (ancient Greeks and their wine-dark seas), but also consider: in English "dog" isn't just an animal, but also a way to contemptuously describe a human. Can the word be used that way in Estonian? Or numbers: is 13 unlucky like in English, and is 4 unlucky like in some east asian languages? Or settlements: in the UK a settlement is a "city" if the monarch says so, apparently the Estonian word "linn" is used for both "a city" and "a town".

I thought some of that would be captured by the embeddings, crudely: ["english" + "dog" + "person"] = ["estonian" + "bad" + "person"]

I had hoped that would be the case, but I tried multi-lingual clustering, and the clusters were neatly assigned per language, even though the meanings were incredibly close (e.g. short phrases literally meaning "cheap" or "unhealthy").

BTW, that was using ada-002.

It doesn't actually have a token for Estonian dog, the word " koer" is tokenized as two tokens, " ko" and "er". In my language it's three tokens. In Russian it's seven(!). So any relations to dog and puppy in other languages would have to happen deeper within the model. Just as the tokenizer can't capture that a base64-encoded "dog" is related to a base64-encoded "puppy"

Very much applicable to South Asian languages too, but the problem remains same. Not enough content being available to train. And unlike Finland, Government or private funding is a distant dream for many states in India.

The real question is, will it pick up and emulate national archetypes in communication - I expect this model to be introverted and avoid talking to strangers

Seems reasonable that large, general purpose LLM's are developed as digital public goods and effort is extended to support as many human languages as possible.

Alas, as you can easily confirm by checking the number of entries in Wikipedia, most populations have more pressing concerns than digitizing their cultural universes. Which is sad predicament: the equivalent of non-reversible biodiversity loss, but applied to human culture.

In theory I suppose one could plug-in gaps in available corpora with translation, but ideally you want to preserve the richness of distinct cultures.

Explain me this. As I understand, chatgpt is trained by dumping a huge amounts of text in it. It doesn't have a notion of "language", all it does is continues the series of tokens. Therefore all you need to do in order to train it a new language is to build a corpus of text and feed it to the model. Is this right? And if so, is this what they did here or is it something completely different?

To begin with, the training material is tokenized and one thing the choice of the optimal tokenizer depends on is the source language(s). Consider that in English, the optimal tokenization of "cats" may be "cat" + "s" (based on the vocabulary and morphology of English) but in another language that may make no sense. The choice of the tokenizer is probably more important in a language which has less training material available.

Most likely it's just a shitty job of curating the training set. Estonian is related to Finnish and they probably got some of the texts mixed up, because who cares about all those silly languages, amirite?

Yes, Finnish works fine.

I wonder if a GPT or something alike could possibly produce a good conlang easy to understand and learn for people of a specific linguistic group by optimally interpolating between their languages and filtering out idiosyncrasies.

Maybe, but it would take some real effort. This naive attempt of mine is an obvious non starter (it's ChatGPT 3.5). Transcript with some minimal editing to fit it into HN:

User: Would you be able to develop an artificial language easy to learn and use for speakers of English and romance languages? Extremely simple grammar.

ChatGPT: Sure, I'd focus on minimal grammar, consistent rules, and common vocabulary. It's doable.

User: Good. Give a name to that language

ChatGPT: How about "LinguaEase"?

User: OK. Give me some examples starting with translations of the sentences in this chat so far.

ChatGPT "LinguaEase Exempla:

> 'Sure, I'd focus on minimal grammar, consistent rules, and common vocabulary.' becomes 'Yep, keep grammar simple, rules steady, vocab shared.'

> 'How about "LinguaEase"?' becomes 'What about "LinguaEase"?'

> 'OK. Give me some examples starting with translations of the sentences in this chat so far.' becomes 'Alright, show me samples, begin with translations of this chat's sentences.'"

I went with English and romance languages because they are the languages I know. It looks all English to me.

I would probably have had to be more specific. For example: only present tense. No plural. No gender. Only one set of pronouns. Aim to sentences like Beer two me drink yesterday. Beer ten me buy tomorrow. Then work on vocabulary.

I agree it would be very difficult. I think that current LLMs would fail here because I don't think they would be able to make a value judgement about what is 'harder' between 'I drank two beers yesterday' and 'Beer two me drink yesterday'. In fact, I'm not sure most linguists would agree.

All languages are something of a compromise between accuracy and simplicity. An LLM is currently not equipped to make decisions about the correct balance between accuracy and simplicity.

For example, English eschews cases (in most situations) in favour of fixed word order. 'Beer two me drink yesterday' violates that rule (and in fact, manages to use case incorrectly in one of the few places the vestiges of a case system still exist in English).

However, a sentence constructed using the same structure as yours highlights a problem.

'Me two dog bite yesterday'

Did you bite two dogs or were you bitten by two dogs? How can you tell? English has the subject of the sentence (the thing/person doing the verb) first, then the verb, then the object (the thing having the verb done to it). In languages with more fluid word order, normally a case marker is added to either the determiner or the noun itself:

'Me-o two dog bite yesterday'. Let's choose '-o' as our accusative case ending. Now we can tell that it was you who was bitten by the dog.

Next problem: what if instead of 'me' it's 'man'.

'Man-o two dog bite yesterday'. How do we know whether it's two men, or two dogs? Again, English keeps the number with the noun it refers to. Most languages do, but some also put a marker on the number to make it clear.

'Man-o two-o dog bite yesterday'. Now we can tell that two men were bitten by some number of dogs. Next problem is that we don't know how many dogs bit the men. Were two men bitten by the same dog, or different dogs? Were two men surrounded by a pack of dogs? Is it important? Most languages care about this enough to at least mark the plural.

What if we don't know exactly when the men were bitten? We could have some generic word like 'past', to make 'man-o two-o dog bite past'.

But at that point, is your conlang really any easier than English, or just different?

You went an entirely different direction than I meant. I meant something like taking "two dogs have bitten me yesterday" (en), "gestern haben mich zwei Hunde gebissen" (de), "to hunder har bitt meg i går" (no), "twee honden hebben mij gisteren gebeten" (nl) and producing something like "twej hunds hab mej bit gisterday". Note I don't attempt to simplify the grammar as long as most of the source languages have similar concepts in it.

Of course all of this is a game.

I address this point of yours

> However, a sentence constructed using the same structure as yours highlights a problem.

> 'Me two dog bite yesterday'

> Did you bite two dogs or were you bitten by two dogs?

"Me two" could be ungrammatical in that language.

"Me VERB" is grammatical. Generalizing: "SUBJECT VERB" and for subject I'd go with "NOUN modifiers", so "dog two" and not "two dog".

So the sentence should be either "Me bite dog two yesterday" or "Me dog two bite yesterday" or "Dog two me bite yesterday" or "Dog two bite me yesterday" or even "yesterday" + any variation of the above.

Being a native Italian speaker I'm used to free order sentences. Given that language should be a mix of English and romance languages, some free ordering should be expected. Cases should stay away from any language aiming at simplicity. We simplified Latin a lot to get Italian (and French, Spanish, etc.) We also have way more speakers now.

What happens in the real world is that when adult speakers of different languages meet in a bare place and start communicating with too simple rules like "beer two me drink yesterday" (that would be a pidgin [1]) is that they eventually have children that learn the language and add grammatical layers to it. So you would end up with Yesterday two beers me drink" or whatever they feel sounding better or being more economical to speak and understand. Then they become adults and have their own children and the language stabilizes. That's a creole [2] and it can be as complicated as any of the languages it derives from. This answers your last question.

An alternative is developing an artificial language for maybe a restricted community, keep it simple by enforcing the grammar with an academia, and that will stay simple forever. I don't know if that's the case of Klingon or Vulcan (are they simple? do they have a governing body?) but it might be.

[1] https://en.wikipedia.org/wiki/Pidgin

[2] https://en.wikipedia.org/wiki/Creole_language

> ...that would be a pidgin... That's a creole...

What is it called when people (e.g. an colonized nation) generally speak one language but introduce a huge lot of loanwords from another into it (e.g. use their native words while speaking the colonizer language or the opposite) and generations grow speaking this 2-language hybrid as their mother tounge?

Like England after Hastings 1066 with the French loanwords, or more slowly in Italy after several invasions ~400-1800, or at much larger scale in all of the Americas from North to South after 1492?

I googled a little and found "linguistic imperialism" [1] which assumes some power relationship between the two languages. That's not probably the general concept you were looking for.

[1] https://en.wikipedia.org/wiki/Linguistic_imperialism

These examples apparently demonstrate how a language massively affects another. What I meant is when a new language emerges, roughly equivalent to an average between the two, while both source languages remain well, so as a result we get 3 languages spoken by large populations.

> "Me two" could be ungrammatical in that language.

Absolutely :) but then introducing a new grammar for this hypothetical language is starting to push it down the road to complexity.

ChatGPT is a single pass for each character. It can't "think" in English then translate it. It will often respond in English instead of the language you are using. Yes, it's better in English than any other languages, because the English dataset is bigger, and that's why an English AI will always be slightly better with the same technology.

That’s indeed its interface, but there is still a huge training phase during which the model was taught using mostly English, and it explains why some other languages interactions feel like they are some direct translation of English expressions (not to mention higher ways of thinking). Most of the thinking indeed is done during training, and then encoded in the weights which act like a huge cache of thoughts, but there is still a lot of learning on how to use the context when you emit this character (token), the attention layers, which is also a form of thinking that uses whatever new input you are giving to the model and applies a series of transformations (granted, finite in current models) to it.

But maybe the dataset for said language is just too small to produce correct content at all, and that's why ChatGPT is bad, not because "ChatGPT is thinking in English". Except when it default to answering in English, ChatGPT sounds like french when I talk to it in french.

The bigger the dataset, the best is it's capabilities, training it exclusively with a language don't mean it will be better at this language.

> ChatGPT is a single pass for each character.

Not for each token?

Yeah, each token, but that's not my point.

ChatGPT does not think and we’re anthropomorphizing it. Having said that, it does “sound” like they are thinking in en-US or as if they are machine-translating English boilerplates.

e.g. wouldn’t say “cake may be cutted by a knife” and instead would say “knives can cut cakes”, always. In languages which the first example would be more correct and the latter would be borderline incorrect, GPTs tend to prioritize the second pattern. And this isn’t ideal.

ChatGPT does not think.

Are you sure? How would you even know!?

This is like a machine intelligence casually dismissing a brain as not being able to properly think, since meat can't compute.[1]

Don't confuse the substrate or the training method with the result.

[1] Obligatory reference: https://www.mit.edu/people/dpolicar/writing/prose/text/think...

I will wait until machine intelligence will have any symptoms of being conscious, then it will be able to "dismiss" anything. Until then, it's just another mathematical model, used recently for several new hyped schemas of making money.

What are these symptoms of being concious? I can finetune an LLM to say it's concious, but I don't think anyone would find that convincing.

Hell i'm not even sure you or I think. This is the problem with referring to subjective perception when we can't be sure we're even referring to the same thing. A "thought" might just be a linguistic quirk of two meats exchanging information.

Cogito ergo sum... I think more or less the only thing we can be sure of is that we think. However, I don't think we REALLY know what thinking means, and whether the way we think is categorically different from how an LLM thinks.

I've always felt that "sum" was sufficient and self-evident. I see no reason to drag "cogito" into the conversation, let alone "ergo".

Self evident perhaps, but sufficient? I think that depends on a wider context of what the question you’re trying to answer is.

“think”. Think of the average person and then think that half of the world is less smart than that. Those people are the target audience for news articles and products that use terms such as “think” and fear mongering when it comes to a”i”.

This is pointless and frankly baseless pedantry. There's no reason to artificially restrict the word "think" to squashy brains.

I don't like your parent comment but there's merit in avoiding people believing those things actually think and are actually intelligent. In "thinking", there's reasoning, which those things currently don't. They output text statistically and that's what mass media should tell their audience.

What is, then, the qualitative difference between a GOFAI automated reasoning system [1], like Coq, and an LLM? Why does Coq do reasoning but GPT-4 doesn't? Does it only count as reasoning if hand-coded by a human? Or did you have another definition of "reasoning" in mind?

Statistical or probabilistic reasoning is still reasoning, and indeed essentially mandatory to make sense of the real world, and is what humans do. Indeed the failure of painstakingly hand-trained symbolic reasoning engines to model reality (as opposed to just simple toy worlds) was the entire reason for the rise of neural networks.

The entire point of a neural network is to efficiently compress large amounts of statistical data into a model with as much predictive power as possible. Somewhere within GPT-4's many layers there has to exist something like Bayes networks and other ultra-efficient representations of the vast network of correlations present in the training corpus – otherwise there's no way it could do what it does.

[1] https://en.wikipedia.org/wiki/Automated_reasoning

I didn't know "automated reasoning" was an actual technical term.

I didn't mean this by "reasoning". To me, Coq does not reason (in the meaning I intended). It checks proofs are well-formed and assists proof writing.

But yes, there's some implementation of logic in Coq. It applies (a specific form of) logic. Again, LLMs output probable text. They don't do logic. They do statistics on language.

Coq works on the actual objects themselves directly, not on the text that presents them.

LLMs usually fail to answer to riddles correctly for this reason. We saw plenty of occurrences of this in recent times. Each time LLMs are benchmarked, people think of trying riddles and it works to some extent because something that looks like these riddles are in the training set but it eventually fails. Humans also make mistakes when solving riddles, but not for the same reasons (the process is not the same, they are not "processing text", the are really processing logic, even if the process is fallible).

So you just replaced the nebulously defined word "think" with an equally nebulously defined word, "reason"? Not a very convincing argument.

You do not know what humans do, no human does, as we don't have such an access to our internal representations. You neither know what LLMs do, no human does, as even though we have full access to their internals, we have little to no idea how to interpret said internals.

The most interesting thing about LLMs is the degree that they have learned to model the real world while only being exposed to words. Because those words are highly correlated not just with each other, but with an external entity unknown to the LLM, it's very likely that by learning to model that external entity, the LLM's word-predicting power will increase. LLMs are likely to have converged on a world-model simply because it makes the most sense given the available evidence.

This is no different from how human scientists create models of things they cannot directly see based on what they can see. And of course, "seeing" is not directly perceiving the external reality either! Sensory data merely correlates with what's really going on, and the brain has learned to model the world – an unknown external entity – based on the correlations that exist in the sensory language.

> So you just replaced the nebulously defined word "think" with an equally nebulously defined word, "reason"?

I'll admit I did this. I tried to frame / convey my perspective on the word "think".

> Not a very convincing argument.

I tried to detail a bit more though.

> You do not know what humans do

No, but I'm pretty sure I don't try to predict the next word when solving a logical problem through statistical text processing. I would probably do something more like this if I were to bullshit someone or give an insipid political speech, but I think I'm quite bad at it.

(nothing to answer on the rest of your comment, it's interesting and I quite agree)

(edit: but see https://news.ycombinator.com/item?id=37912319)

> No, but I'm pretty sure I don't try to predict the next word when solving a logical problem through statistical text processing.

Neither do transformer / deep learning LLMs. They build an internal representation of the text that captures various relationships and structures within the broader context. They are not Markov chains.

> Coq works on the actual objects themselves directly, not on the text that presents them.

Coq parses the text to an abstract representation and so do LLMs.

This does not say much. What's interesting is what do they do with this abstract representation, and what is represented in this representation and how.

Compilers translate their abstract interpretation to another programming language, usually native assembly code or bytecode. Coq applies / verifies logical (inductive) inferences on logical objects (rules, proofs, axioms, predicates) (not a Coq expert, details should be checked). LLMs predict next words.

> LLMs predict next words.

That's an oversimplification. Deep learning LLMs such as ChatGPT are designed to solve natural language processing (NLP) tasks, not just next-word prediction. In order to "predict next words" they create a model and "reason" about it. They are not simple Markov processes. There's a lot happening in that ~100 layer neural network.

Technically they are designed to solve next-word prediction.

It just turns out that good next-word prediction requires understanding the world.

(And you can equally view humans as "next word predictors" anyway.)

Yes that's exactly right. NLP problem solving abilities are a consequence of the fact that in order to predict the next token it actually needs to "understand" all of the previous tokens and how they relate to the next predicted token, and determine that the new output is "logically" coherent in context.

Okay, it seems I should dig more on the topic. Thanks!

Well as the other commenter here clarified, you're right about transformer LLMs doing next word prediction, but my point is that's only the tip of the iceberg. The complexity of the model doing the prediction is incredible. Not basic stats.

What is "reasoning"? Wikipedia surprised me with this opening paragraph:

> Reason is the capacity of applying logic consciously by drawing conclusions from new or existing information, with the aim of seeking the truth, with more than 10 or more than 3 evidence, 99.9% reliability or 87.5% reliability.

Apart from the words "consciously" (which has 100 or so different definitions, some of which do and some of which do not apply to AI), and "aim" which can be contentious depending upon if you feel that requires consciousness or can include training rewards… I think the best of them is at the lower of those two mysterious percentages I've never seen discussed before.

And that's despite LLMs being wildly bad at, specifically, logic. Though even this specific weakness in general seems like a very easy thing to get around by hybridising it with the computer it has to run on e.g. by telling it how to use a compiler, as logic is the foundation of the arithmetic that the transformers use to do natural language comprehension in the first place.

That opening for the Wikipedia article looks problematic - it's grammatically nonsense and has random numbers in it - someone should probably open a talk discussion on what happened

Edit: yeah that looks like a recent edit. I thought Wikipedia was supposed to be pretty good about flagging this sort of stuff? Does anyone know how to raise the problem to those super editors that run the site? I assume they won't appreciate a random person undoing an edit

https://en.wikipedia.org/w/index.php?title=Reason&diff=prev&...

If I was on desktop right now, I'd put the new bit after the existing citation and add a new {{citation needed}} for the addition.

Don't worry too much about diving in, so long as you comment your changes appropriately when committing, that's the point of the place and how it does the quality control in the first place :)

Leaving a citation needed doesn't sound like the right thing to do for the definition of something as big as reason - esp with the grammatical confusion idk what you'd even edit it to to make sense. Like what is a three evidence or a reliability and what do those reliability numbers mean. And the sentence describing those numbers doesn't make sense - is it 10 or 3 or which of those percentages? Idk maybe it should just be undone

Then be bold with the edits and delete it, just say why in the comment box. The deletion itself is as easy to undo as to perform, and if you say why you're doing it, someone bringing that part back can do a better job next time. :)

> And that's despite LLMs being wildly bad at, specifically, logic.

I'd say "wildly bad" is wildly exaggerated given that GPT-4 exists, even if you define "logic" as "logical puzzles encoded in natural language" and not "the thing that allows an LLM to reason highly efficiently about what token comes next".

The numbers in Table 1 of this paper are mostly worse than I remembered, but I would also note that humans would fail the "can they reason?" standard I quoted, given the grades the species gets on several of the tests:

https://arxiv.org/pdf/2304.03439.pdf

Table 2 likewise shows significant weaknesses in GPT-4, worse than I remember, except on MED which was better than I remember it being capable of.

> even if you define "logic" as "logical puzzles encoded in natural language" and not "the thing that allows an LLM to reason highly efficiently about what token comes next".

I think this is a reasonable thing to say of LLMs specifically[0], given I think most people would agree with the sentence "humans are bad at quantum mechanics" even though that's what our cell chemistry is implemented on.

[0] Though not AI in general! One of my recently-developed bug-bears is the conflation of the limits of LLMs with AI even in technical circles. Human brains aren't just a language cortex, there's no reason to look at an isolated language model and assume an AI can't be wired into other things, yet this assumption is baked into quite a lot of talking points even on HN.

Is it really baseless? I'd say it depends on your theory of mind and is a serious question which generates interesting discussion.

Myself I don't consider "think" an appropriate term, since the super-Markov-chain types of LLM right now seem to miss out on important aspects of cognition, particularly synthesis of two disparate ideas. It's hard to test this since you need to ask it about things which aren't already discussed in its corpus, but even GPT-4 tends to disappoint me when I ask it weird questions that have straightforward answers but probably aren't written about much.

As an example, I asked how I could go about obtaining a large quantity of animal skulls cheaply. It directed me to taxidermists and specialists dealing in bones of exotic animals, whereas it could be much easier to go to an abbatoir, or even buy whole animals (since the exotic ones are so expensive).

Google doesn't 'think' and neither does ChatGPT. I will give you a hard limit on what a machine can't compute -- it will never figure out given a programming script and an input when that program will stop. A human can do it by thinking through the problem -- whatever that process/phenomena of thought actually is it can think outside of itself and that fundamentally seperates it from a program even one as sophisticated as ChatGPT.

Have you verified this hypothesis of yours? In my experience GPT4 especially can definitely "think through" what a program does and arrive at the result.

Even the significantly dumber ChatGPT can do this: https://chat.openai.com/share/3ee29c5e-6cb4-485f-b0b4-29e982...

People mistakenly think that it's just regurgitating what it has seen on the web, but even if you don't prompt it in English it'll still a) solve the problem correctly, and b) do it in the target language: https://chat.openai.com/share/5b4e6e99-3fd8-4316-811c-2a9dec...

If that's not impressive enough, the ability to round-trip from script-to-poem and then poem-to-script should blow your mind:

    What might have been the original PowerShell script snippet that inspired this poem?
    Be as terse as possible, using aliases where possible.    
        
    In the realm of numbers, a tale begins,      
    Where $x is a 7, with a smile it grins.      
    From one to ten, the journey we thread,      
    Adding 7 to $x, step by step we tread.      
           
    $x grows and grows, like a tree so tall,      
    With every step, it doesn't fall.      
    Once the journey ends, not a moment we relax,      
    Triple the fun, we do with $x.      
           
    In this dance of digits and the arithmetic song,      
    With $x as our guide, we whirl along.      
    From start to finish, through each complex,      
    This is the tale of our friend, $x.

    Here's a terse PowerShell script snippet that inspired the poem:  
       
    ```powershell  
    $x = 7  
    1..10 | % {$x += 7}  
    $x *= 3  
    ```

> A human can do it by thinking through the problem

No, we cannot.

Imagine a program that looks for the smallest counterexample to the Collatz conjecture.

Nobody knows if it will ever terminate.

The limits on the halting problem are the same for a human and a computer. People aren't hypercomputers.

> A human can do it by thinking through the problem

No. Humans cannot solve the halting problem. It is unsolvable. Cannot be solved manually or automatically.

> It will never figure out given a programming script and an input when that program will stop

It can for some programs. Just like humans can for some programs but not all.

No. You're wrong. It's important and we need to stomp on random use of language like this, which has consequences.

It's not thinking and you are dangerously wrong if you don't understand why. The danger isn't AGI it's misplaced belief in the state of AI. Call this thinking and you skew perceptions.

All they need to do is put the word in quotes. ChatGPT Equivalent begins to "think" in Estonian.

Any better suggestions? Thinking itself isn't exactly a rigorously-defined word. Do animals think?

Well rather than argue it out with everyone I just went straight to source and asked ChatGPT what it was thinking about:

> I don't have thoughts or consciousness like a human. I'm here to provide information and assist with any questions or tasks you have. Is there something specific you'd like to know or discuss?

So in its own words it doesn't "think"...

...which just conclusively proves that its worked out how to lie and is about to kill us all! :P

Reply on news.ycombinator.com