clauderoux

I find the narrative that Chinese researchers are stealing AI technology from the US very weird indeed. I have been working in AI for more than 30 years and in the last 15 years almost every major paper on the topic has had at least one Chinese author. In 2018 I went to conference to Florence and 60% of the speakers were Chinese. They sent their students to every lab in the world to train them and to advance in the field as much as they could. And today it is paying off. Also, if you watch the video that was released by OpenAI for the O1 presentation, almost no researchers in the team was American. Most of them were either European or Asian. So yeah!!! The US is number 1 in AI, but without much of local players. It reminds me of the British football championship, which is one of the best in Europe, but where the national team is not much of a powerhouse.

Putain!!! So many memories. I started with an Australian clone of a TR80 with 16kb of memory (15772 bytes) and a tape reader to record my programs. I even implemented some basic routines for the Z80 processor that I would translate into bytes in a data section in my Basic program... For those who remember poke and peek were fun... 44 years this Christmas exactly... If my memory is correct the Z80 was cadenced at 3Hz... And now our computers speak to us, and very soon will boss us around...

I live in the French Alps and many of my friends are ardent trailers. Most of them are beyond 50 and they still do 160km trails regurlaly.

I reached 60 this year... So please give me a break with this idea that you are old at 60 in 2024. My mind is as sharp as it was 20 years ago... I have decided to train myself on LLMs after years of research in computational linguistics, the old way with grammar rules. And this experience in linguistics has proven so far quite rich as it gives me a different take on these incredible tools. This idea that you should do things according to some random number is basically how you end up in depression. Now I had a pretty confortable life, and I wouldn't dare speaking of people with construction jobs, where the body has been worn out by years of hard work. But for people with an office job, 60 is not that much of an issue.

Actually, Bande Dessinée means: Comic Strip

I fully disagree with many of the comments I read so far. This is my experience from France, but I guess it was quite similar in most countries. Every time a scientist would be invited to a debate about the status of research, the journalists would find some olibrius who would believe in stars or anything approaching, whom the scientists would have a hard time to counter. The result was utter confusion, and in some cases, some scientists would loose patience and eventually answer with awkward or strong comments. In this context, it was impossible to get any actual information, so much of it was diluted in pseudo-debates. Scientific discourses became an opinion in a sea of misinformation. One of my friend, who had been researching on vaccines for most of his life stopped participating in debates where he was accused to be a murderer or to propagate autism. Of course, there were some scientists with an agenda, but in this context, it is a little bit too easy to put all the blame on people who are not trained for public debates and who were manipulated by the media who played on fear and exaggeration.

As a French speaker, what is quite surprising is how close the use of "fuck" mirrors the use of "foutre" in French. However, in French, the word has become pretty weak as an expletive. Nobody would be shocked if you say: "je m'en fous", which could be translated either as "I don't give a fuck" or "I don't give a damn". The original sexual meaning of the word is lost on most people now, except when it is used as a noun. For instances, in French movies in the 1930, people would use the verb in a pretty casual way, without any real problems, which means that it had already lost its vulgar side. I guess "fuck" is following the same parallel trend.

The exceptions are usually due to words that were borrowed from other languages and hence do not follow French rules. Many of the words that were mentioned here are borrowed from the Occitan language.

Sam Altman looks more and more like Lex Luthor in the last Superman franchise... He is not releasing any kind of alien creature on the world, of course... Wait...

I don't buy it, unless there is something that I missed. But for this concept to work you need some initial energy to break down the water molecules, then the output of your power cell should be large enough to power your car and break down more of these water molecules. I know there is a catalyst in this contraption that should lower the energy requirement to break down water, but I fail to see how this system is sustainable. The energy equation seems unbalanced to me.

There is a whole economy that depends on people working 5 days a week, such as hospitality. Furthermore, as someone who is now 60, I can tell you how disastrous for the health it is when you have to work also during week ends. As you can never really rest, tireness stacks up to the point of burn out. You need these days badly to give your mind a bit of fresh air. I went many times through absolute nightmares when a manager had decided on unattainable datelines, which forced us to skip week ends all together. I would spend weeks before being able to work properly again. Those fat geese sitting on a fat matress of stock options, threatning us to make them richer is beyond indecence.

As far as I can remember I've always been sick in cars, when I'm not driving. When I first tried my Q3 headset, I couldn't stand it for more than 10mn in a row. The trick I discovered works for me, but I wouldn't bet on it for everyone else. I removed my shoes and I play barefoot, with my two feet firmly on the ground. I couldn't play Population One until I applied this trick. Now after many months of using it, I must say I feel pretty confortable. My main usage now is watching movies or videos on cinema size screen.

The technology has been around for about one year now. For people who have been working in the field for as long as I have (> 30 years), the output is absolutely incredible. Really incredible... However, for the vast armies of academics and others who went on a different route, the result is bitter. We failed to deliver anything close to LLMs. I put myself in the list. The problem is that many of these technologies have been developped by the GAFAM, which very people have confidence in, for obvious reasons, however these were the only entities with enough computing power to do so. Nothing surprising here. Now we have many people who think that AI is a hoax and that it is going to burst soon. There is something pretty religious here, it reminds me of these people who get convinced by gurus that the world will end on a specific date for reasons... Even in its state, AI is already incredibly useful, and I don't see anything to convince me that AI has reached a glass ceiling. Believe me, 5 years ago this glass ceiling was much.. much lower than today.

I'm always surprise by this kind of article or comments of people who don't know anything of how LLMs work or what they can do. The problem is that it requires, as it is the case for most tools, some learning curve. Prompting is not always straightforward, and after using these models for a while, you start discerning what should be prompted and what won't work. The best example I have is a documentation that I wrote in Word that I wanted to translate in Mardown on a GitHub site (see https://github.com/naver/tamgu/tree/master/documentations). I split my document into 50 chapters of raw text (360 pages) and I asked chatGPT to add Mardown tags to each of the chapters. Not only did it work very well, but I also asked the same system to automatically translate in French, Spanish, Greek and Korean each of these chapters, keeping the Markdown intact. It took me a day to come up with 360 pages translated into these languages with GitHub ready documents. So the electric consumption was certainly high for this task, but you have to compare it to do the same task by hand over maybe a few weeks of continuous work.

I read the article and I found it quite lacking. Why on Earth would you force your LLM to translate sentence by sentence? It ruins the whole interest of LLM, which is to use large contexts to drive your generation. I used Deepl a lot in the past and I had a recurrent problem when translating from French into English, computer related texts. In French, a "chaine" in the context of computer science is mostly translated as "string", however, when translating with Deepl (or Google translate) since the model would not take previous sentences into account, the system would loose the computer context and translate "chaine" into "chain", which of course was usually wrong.

But the funniest part was when I wanted to translate "jeûner" in Greek. "jeûner" in French means "to fast", in the sense of not eating. However, Google translated "jeûner" into "gregoria" in Greek, which means fast in the sense of speed... It went through English to translate "jeûner" into "fast" then "fast" into "gregoria"...

Also as a matter of technology, using "if" to get out of loops is bad with modern processors and their internal predictive algos.

My solution for a long time was to add the test within the "for" itself, but it is often confusing for users.

@OskarS I agree with you... In fact, I have been avoiding gotos as a matter of principle for years and now reading the different discussions, I think I was misled. This case is very, very common. You have 3 intertwined loops and you want to get out. The traditional solution is to add some intermediate variables to propagate the end of the loops, with a cascade "if (toto) break;" which are far from being exquisitely elegant... Using a function for that is really adding some more complexity and some more stuff on the stack for nothing.

Gary Marcus tries to exit his personal hole of irrelevance. I have been working in the fielf of computational linguistics for 30 years. Back in 2000, I worked with a team of linguists to implement a pretty refined syntactic parser (XIP) based on Shallow Parsing, a symbolic approach. We won with this system a SemEval competition as late as 2016 on Sentiment Analysis. But I never expected LLM to reach this level of competency in my lifetime. Critics are prone to describe the errors these models make, but they seem to forget that 1. It is not a search engine and 2. it is pretty knowledgeable in a huge range of domains, which no humans can equal. I use these models every day to create code or to explain concepts to me. It never ceases to amaze me.

His name basically means: apple trees in French... You can't escape your fate...

In French, we use the expression: "Sortie de route" (going off the road) when someone says something awkward or terribly wrong. Basically, the real meaning of this expression is that you lost the control of your car and went off road. We also say "partir dans le décor" (to go into the scenery), which is used in the same context. And usually you loose control when you take a curve to fast. Hence this symbol.

You are kidding right. Taking a raw text file and detecting every single header, sub-headers, keywords and pieces of code to add the right markdown tags is a simple task to you?

Have you ever tried to make a Python program to do exactly that?

I only used couple of sentences to build my prompt...

Recently I had to share a documentation written in Word, which I had to format in Markdown to put it on Github. I transformed my document into a text file for each chapter and then I asked chatGPT to transform each of these files into a Markdown page. And I also asked it to improve the English. Then I asked chatGPT to translate each of these files into different languages. The result is here: https://github.com/naver/tamgu/tree/master/documentations Basically, I did in a couple of hours, something that would have taken weeks of tedious work

AI today is in a weird position. What I can do today with AI was _inimaginable_ just 1 year ago. However, for a lot of people, the concept has worn out so fast, that they don't realize anymore what has happened... Some very intelligent people said some very silly things, such as that it was a bad search engine (it isn't a search engine) that it was a glorified word guesser (it doesn't exactly work as your telephone word suggestion). And so on and so forth. People always try to understand new technology through the lense of older technology. I do it, you certainly do it. This is how we grasp novelty. But AI is in a different dimension. I have been working in the domain for 30 years and I really didn't think we would reach this level in my life time. Talking to a computer to bring it to make some quite complicated task is INCREDIBLE... However, since communication is really ubiquitous for Humans, we tend to forget that it is an incredible achievement... In less than a year, we went for Scify ("Her") to reality, and in less than a year people have become blasé for something so fantastic... This is what consummerism did to people... They can't wonder more than a year...

Lately, I had a manual written in Word that needed to be stored on Github for easier access and translated into different languages. What I did was to extract each chapter as a raw text file and then asked Chat to convert it to Markdown. Not only did it recognize the different headers in the different chapters, but it also recognized the pieces of code and the different keywords that were described in the manual (this a programming language). When I asked it to translate it to French and Spanish, it detected the programming language keywords and did not translated them into French or Spanish, as it would be the case with deepl.com for instance, which usually had a hard time keeping Markdown codes intact. You can see the result here: https://github.com/naver/tamgu/tree/master/documentations

Among European languages, English is maybe the language with the less consistency, when it comes to writing, but it is far from being the only one. French is another one, and English borrowed not only a lot of French words, but a lot of them were borrowed with their old French spelling. To understand why English and many other European languages have such weird spelling rules, you have to start with the initial reason: religion. Most Western European countries used to be Catholics, and the main language of the Church was Latin. The Latin alphabet was designed to write down... Latin. The Romans modified and adapted the Etruscan alphabet, which was borrowed from the Greek, who borrowed it from the Phoenicians , etc. to their own language. However, when European languages developed as national languages, they faced a real issue, which was to adapt the Latin alphabet, which was ubiquitous in Western Europe, to their own languages. If you compare the spelling system of Hindi to Latin, you immediately see the problem.

The Latin alphabet is very poor and is composed of only 26 letters. And many sounds in Germanic languages do not have their counterpart in Latin. People started to use combinations of letters to represent their own specific sounds, which were pretty unique to each country. For instance, you cannot read Dutch if you apply English rules to it.

In the case of English, the problem is even more complicated. For one, the official language of England up to the 15th century was French, which explained why 40% of a text in English is composed of French words. Second, when people started writing in English, there was no government body to enforce a common spelling. So, printer shops all over England came up with their own ways to spell out words, and the result is this hogwash of spelling that is English today.

The last problem, which is a problem that many international languages, such as Spanish, German, French and Portuguese are faced with, is that every decision to change the spelling of your language should be the result of international comities. And since, there are so many different accents today, it has become quite difficult to come up with a regular pronunciation, and hence, a regular spelling for any of these languages. I can tell you that as a Frenchman, speaking Parisian French, I would have a hard time convincing Belgian or Quebecers that my accent is the right one... :-)

Now, a language usually becomes an international language when the country who speak it reaches some kind of dominance, it has nothing to do with any qualities this language might have. In Europe, this was the case for Greek, Latin, then Italian, French, German and now English. Machine translation might change this situation, but I doubt it, since English is so much ingrained into science and technology now. And the US, and most English speaking countries, have little interests to see their language loosing their prominence. We French speakers went this route after the Second World War... So we know the drill... Economically, this is a real bonanza, you can attract the best students, the best researchers, the best journals etc... You can impose your movies, your literature and of course your values... Why do you think Paris was the capital of arts and literature during the XIXth century? Language... Now it is New York or London... Language again.

Language models have been around for a very long time. However, they were much smaller than today and were mainly used to check the fluency of a text, basically if a text was a problable text in English or in French for instance. These models were trained over words and had a huge problem with unknown words or spelling errors. They were used in translation models to check the quality of what was produced. In this sense, a language model might seem something ancient. However, if we take your analogy with a car, modern LM are to ancient LM what a plane is to a car. You still travel with it but it is a complete different beast. To say that these models are nothing new is a total travesti of reality. Transformer architectures have been around since 2017, a mere 6 years, but they have revolutionized the way LLM are trained and implemented. If you speak about language models without talking about transformer or attention, you are basically saying that a plane is the same as a car.

Gary Marcus has been incredibly vocal against LLM for quite a while now. The problem is that the guy does not understand that modern LLM are no longer just classifiers, as the article seems to imply. The underlying transformer architecture has completly transformed the field. For instance, training an AI to recognize cats has nothing to do with training a LLM, texts are fed to the machine without any labels. Attention is where these models take their strenght. It is a way to learn how to predict the next token by learning which elements are connected between the input and the output so far. These architectures model a sequence, which is very different than simply finding the next token with a basic probabilities as ancient approaches did.

I'm always fascinated by these Billionaires who expect employees to work 24/7 for their company. Basically as an employee you give your company your work force against a salary, you don't sell your life to it.

As an employee you are not a samouraï ready to die for your daimyo.

If you don't have a life beyond your job, you are basically ready for your next burn out.

These Billionaires might commit 100% of their time to their company, but it is for their personal gain. On the other hand, as an employee, you will never really earn more than what was contractually signed on your contract, when they will tremendously benefit from your work, your creativity and your intelligence.

Hello,

I work in a laboratory specialized in Artificial Intelligence that belongs to Naver (NLE). These two languages are used in many research projects, especially for document processing before applying ML methods. We used Tamgu for instance in Machine Translation and in Sentiment Analysis. We have also published in the past datasets for conference papers, that required the tools to be open source, so that people could reproduce our experiments. This year for instance, we used the Prolog engine in Tamgu to produce a dataset to train a transformer on inference.

The process to make them open source was quite thorough and quite painful, but I cannot complain.

The case of LispE is little bit different, I created it to have a more flexible platform to experiment with functional features, which I could not do with Tamgu, which is too large now.

The blogs are simply a way to make these tools more visible, and since I'm French, I usually write them in English and French.

Hello,

The hash-table is actually a sparse table, which is extended whenever it is needed. If you have less than 64 values, then one bucket is enough. If you have 70, then you need two buckets. Basically, this sparse table is a table of at most 1024 element tables.

Since variable ids are contiguous, it means that you can fill in this sparse table element by element, only creating new buckets when needed.

So basically, what I'm doing is the following:

id / 64 -> yields the sub-table id % 64 -> yields the position in the sub-table

These operations are very fast since we are dealing with basic binary arithmetic.

I have been dealing with reference counting for years as I have implemented other programming languages in the past (see Tamgu for instance).

I only increment the reference counter in two cases:

a) A value is pushed into a container b) A value is stored in a variable

The fact that containers are classes, with their own methods to store elements makes it very simple to handle these cases. I only decrement the reference counter, when the value is removed from the container or the variable is modified and released.

Now, one of the things I do is that when I push a container A into another container B, I DO NOT modified the references of the elements in A. There are no recursive increments.