matusp

Well I don't see people running their web search locally, so I don't think they will run their own search+LLM.

Scaling works, the problem is that it is practically impossible to scale much. There is only so much energy, text data, GPUs, etc. The folly of scaling is that we are living in a finite world. The huge investments in AI for the past few years are probably hitting the practical limits of scaling for now.

I also feel like most insiders were fully aware of this fact, but it was a neat sales pitch.

The sota chatbots are getting more and more functionality that is not just LLM inference. They can search the web, process files, integrate with other apps. I think that's why most people will consider local LLMs to be insufficient very soon.

This is a natural response to software enshittification. You can hardly find an iOS app that is not plagued by ads, subscriptions, or hostile data collection. Now you can have your own small utilities that can work for you. This sort of personal software might be very valuable in the world where you are expected to pay 5$ to click any button.

It's not really a mystery why it happens. LLM APIs are non-deterministic from user's point of view because your request is going to get batched with other users' requests. The batch behavior is deterministic, but your batch is going to be different each time you send your request.

The size of the batch influences the order of atomic float operations. And because float operations are not associative, the results might be different.

Not convinced. There is an obvious value in having more food or more products for almost anybody on Earth. I am not sure this is the case for software. Most people's needs are completely fulfilled with the amount and quality of software they already have.

ChatGPT - 5.8b visits - -5.2% MoM

Gemini - 1.4b visits - +14.4% MoM

Yeah, ChatGPT is still more popular, but this does not show Gemini struggling exactly.

Do we have a better estimate? I don't think it's particularly difficult to get information from the occupied territories, the people there seem to freely use Internet.

It's my understanding that this war is really not particularly bloody for civilians as it is moving so slow that Russians are taking month to conquer pretty small towns and cities and the civilians can usually evacuate or hide. The bombing campaign has some civilian casualties, but I mostly see headlines mentioning <5 dead overall per occasional huge wave of drones and missiles.

My skills are alright, nothing too crazy. I've had messages that were more spammy/scammy, but the volume was not crazy high, they were usually fairly obvious, and I resolved by simply ignoring them. I would say that the random chance of getting an interesting job offer, however small, is probably worth it for most professionals.

However, one thing I haven't mentioned is that I am based in Europe. My small sample size of people reaching out to me is showing that US contacts are usually less serious (e.g. ghosting). Maybe the US experience is so much worse overall due to this?

My current job cold contacted me via LinkedIn. I use LI minimally, basically only to establish connections with my network, and it already gave me huge value back.

Working as ML engineer/researcher:

- LLMs are absolutely abysmal at PyTorch. They can basic MLP workflows, but that's it more or less. 0% efficiency gained.

- LLMs are great at short autocompletes, especially when the code is predictable. The typing itself is very efficient. Using vim-like shortcuts is now the slower way to write code.

- LLMs are great at writing snippets for tech I am not using that often. Formatting dates, authorizing GDrive, writing advanced regex, etc. I could do it manually, but I would have to check docs, now I can have it done in seconds.

- LLMs are great at writing boilerplate code, e.g. setting up argparse, printing the results in tables, etc. I think I am saving hours per month on these.

- Nowadays I often let LLMs build custom HTML visualization/annotation tools. This is something I would never do before due to time constraints, and the utility is crazy good. It allows my team to better understand the data we are working with.

The interesting question is how much more software we actually need. Will software be done one day, all built up, similar to railway networks? With LLMs, software engineering might get cheaper, but it can also lead to increased demand. Resource getting cheaper actually very often leads to demand skyrocketing, as it becomes accessible to new markets.

So if we could set it up in a way where there is no slippery slope, you would be okay with it?

AI alignment is not a solved problem by any means. As long as LLMs hallucinate, they cannot be considered aligned. You can only be aligned if you have a zero probability of generating hallucinations. The two problems, alignment and hallucinations, can be considered equivalent.

Just to play devil's advocate, you're okay with forcing a criminal to sit in a room for the rest of their life, but you're not okay if they also have to work for society during that timeframe. What is the main argument why the first case is okay and the second is not.

1. Go out every morning to work in your field. 2. When the Sun rises, make a note on the same fixed piece of wood, e.g., a fence. 3. Observe the leftmost and rightmost positions, these are your solstices. 4 You can now use your fence to identify and predict solstices.

The AI hate is unreasonably strong right now. People are acting like adding one feature they don't like or need to a browser is a borderline critical offense because it is an AI feature. I find it shocking how quickly the public in the US/EU developed this sort of hate towards one of the most interesting technology of the last decades.

Does cooling destroy H20 molecules somehow?

They don't have any more juice left to squeeze at that front. The lack of new ideas in LMs is pretty palpable by now. There is a bunch of companies with billions invested in them that are all just looking at each other, trying to figure out what to do.

I don't really buy this post. LLMs are still pretty weak at long contexts and asking them to find some patterns in data usually leads to very superficial results.

> Less than half of current AI projects had generated more in returns than they had cost, respondents said.

Compares to non-AI software, is this a bad number?

In general, I find it not convincing to use patent-based metrics as the only argument. The parent writing culture might have changed in various ways that are not related to the quality of the ideas. This undermines the entire argument...

I would say it was discovered, not invented. People were messing around with some algorithms, intruiged by their results. Eventually researchers discovered that with using certain training algorithm with certain data can lead to really wonderful outputs. But this is pure empirical discovery.

No AI researcher from 2010 would predict that transformer architecture (if we could send them the description back in time), SGD, and Web crawling could lead to a very coherent and useful LMs.

What do you mean? I can only input k tokens into my LLM to calculate the probs. That is the definition of my state. In the exact way that N-gram LMs use N tokens, but instead of using ML models, they calculate the probabilities based on observed frequencies. There is no unbounded context anywhere.

LLMs definitely also have finite context length. And if we consider padding, it is also constant. The k is huge compared to most Markov chains used historically, but it doesn't make it less finite.

What do you mean? The states are fully observable (current array of tokens), and using an LLM we calculate the probabilities of moving between them. What is not MC about this?

LLMs are indeed Markov chains. The breakthrough is that we are able to efficiently compute well performing probabilities for many states using ML.

You can achieve medieval peasent level of life quality by not working at all these days.

This was common even in 2015. You can still see people removing stop words from text, even when they feed it to LLMs. It's of course terrible for performance, but old habits die hard I guess.

Consider using structured output. You can define a JSON with specific fields, and LLMs are only used to fill in the values.

https://ai.google.dev/gemini-api/docs/structured-output