fraboniface

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

One could also see it this way: shouldn't the goal of society be to allow more and more people to do things for the beauty of it rather that producing ever more useless stuffs and destroy the planet on the way?

Thats' why you need light EVs. Norway has electric tanks. China has light EVs.

We'll have won politics the day people understand that individual responsibility doesn't work at scale, and that, on average, people just respond to incentives.

You're dead right, it would be the one killer move to remove a lot of perverse incentives, fix the internet, possibly even social media, and all live in a happier world. The whole economy would stop paying the ad tax to Google and Meta.

And it's not that impractical : just make a consumer-run search engine for products and services.

> Developments to the model architecture contribute to the significantly improved performance from previous model families.

I wonder how significant this is. DeepMind was always more research-oriented that OpenAI, which mostly scaled things up. They may have come up with a significantly better architecture (Transformer MoE still leaves a lot of room).

While "quality of life first" remains true in spirit for a lot of French people, this hasn't been supported politically since 2000 (when legal weekly work time was reduced to 35 hours - many people do more but they are compensated for it). And even that law was an exception. In truth, France has taken the neoliberal turn of the 80s almost as much as other countries, and growth and competitiveness has been the only mantra of governments for 40 years. We're mostly protected by laws passed before the 50s.

How do you (especially Europeans) understand this move from Russia? Can it really sustain a war against us? Does it want to break NATO by proving the US won't move?

"The World Beyond Your Head" from Matthew Crawford is exactly about this. Definitely recommend reading!

On this, read Daniel Susskind - A world without work (2020). He says exactly this: the new tasks created by AI can in good part themselves be done by AI, if not as soon as they appear then a few years of improvement later. This will inevitably affect the job market and the relative importance of capital and labor in the economy. Unchecked, this will worsen inequalities and create social unrest. His solution will not please everyone: Big State. Higher taxes and higher redistribution, in particular in the form of conditional basic income (he says universal isn't practically feasible, like what do you do with new migrants).

They mention that chronological order increases the amplification of extreme content. They don't seem to have tested only subscribed content though.

Fossil (in particular coal) thermal plants that were planned for shutdown are being kept online or restarted because of AI energy use. Tech had a pretty minor environmental footprint until now but it's growing rapidly due to AI, for use cases that are clearly not vital and for a good part garbage.

An idea I had is organizing week-ends in big houses with maybe 30 strangers of similar age. You make them have a great time, and afterwards you invite them to indicate the people they really liked. Over time you can make them see each other again and maybe do some recommendation engine to guess who in the community they will like but still haven't met.

The problem is that fossile fuel companies (and other polluting industries) don't just answer an existing demand but also influence on that demand: they lobby against environmental regulation, run influence campaigns against climate science, use greenwashing, etc. Those companies and their shareholders can't be taken out of the equation as they're not neutral.

Exactly, every country should urgently have a public debate on how best to use that technology and make sure it's beneficial to society as a whole. Social media are a good example that a technology can have a net negative impact if we don't deploy it carefully.

It's very hard for parent to resist the insistent demands of their children to get a smartphone when their classmates all have one, and not having one means being or feeling excluded. This is the kind of coordination problem better solved by regulation.

Creation is the single most fulfilling human experience after having children (which is also creation). I'm not sure we want to take that away from us.

Why ignore academics?

Very interesting, I've been thinking about this kind of approach but haven't had the time to really work on it. So what kind of business model do you have? Is it a kind of drop-in replacement for vector dbs?

Out of curiosity, if it's not a trade secret, how do you plan to handle conflicting data (two sources saying different things on the same topic/data)?

Wow nice, what was the service to attract people so well?

This has indeed been a big blocker, but the article is proof that the market now wants it. And there are plenty of new nuclear startups, as well as interested researchers. What was lacking so far was a real need energy-wise (in the absence of a high carbon tax) and regulatory stability/transparency.

Edit: also, sodium fast reactors have existed for 20 years. The R&D has mostly already been done for that tech. But the lack of projects make it stuck to TRL 8.

I don't really understand this argument. How are dozens of nuclear power plants a single point of failure? Because of the uranium mining and processing? Then we can invest in fast reactors, which consume 100x less and require less R&D than renewables have benefited from in the last decades. Or seawater uranium, another 100x in reserves, distributed all around the world.

Videos streams from everyday life sound like extremely high quality data for training AI. Videos available on the internet are a very biased sample compared to that.

Not exactly, they use a small but long-context model that has the whole dataset in its context (or a large part of it) to generate the chunks as elements of the reply, before passing those to the final model. So the retrieval itself is different, there is no embeddeding model or vector db.

Maybe I was unclear (not a native speaker) but that's what I meant: good luck designing a social media without those. So yes, I agree with you.

> We know parents want to feel confident that their teens can use social media to connect with their friends and explore their interests, without having to worry about unsafe or inappropriate experiences.

Or how to ignore the fact that the problem with social media is not harmful content, it's the social media itself. The addiction, the loss of time, and the constant comparison with others. Go design a social media without those.

Some commenters seem a bit confused as to how this works. Here is my understanding, hoping it helps clarify things.

Ask something to a model and it will reply in one go, likely imperfectly, as if you had one second to think before answering a question. You can use CoT prompting to force it to reason out loud, which improves quality, but the process is still linear. It's as if you still had one second to start answering but you could be a lot slower in your response, which removes some mistakes.

Now if instead of doing that you query the model once with CoT, then ask it or another model to critically assess the reply, then ask the model to improve on its first reply using that feedback, then keep doing that until the critic is satisfied, the output will be better still. Note that this is a feedback loop with multiple requests, which is of different nature that CoT and much more akin to how a human would approach a complex problem. You can get MUCH better results that way, a good example being Code Interpreter. If classic LLM usage is system 1 thinking, this is system 2.

That's how o1 works at test time, probably.

For training, my guess is that they started from a model not that far from GPT-4o and fine-tuned it with RL by using the above feedback loop but this time converting the critic to a reward signal for a RL algorithm. That way, the model gets better at first guessing and needs less back and forth for the same output quality.

As for the training data, I'm wondering if you can't somehow get infinite training data by just throwing random challenges at it, or very hard ones, and let the model think about/train on them for a very long time (as long as the critic is unforgiving enough).

That would be very bad product design. My understanding is that the model itself is similar to GPT4o in architecture but trained and used differently. So the 5x relative increase in output token cost likely already accounts for hidden tokens and additional compute.

Sorry for the additional unwanted advice, but have you considered a pay-what-you-want pricing for the upfront fee?

You can set the minimum to zero and a default above your target, and perhaps ask only after a short free trial period. I heard that it often yields better results than we would expect.

This could explain why we naturally perceive and talk about an empty gaze or intelligent eyes. We all have a sense that someone's eyes somehow give a clue about their intelligence. Perhaps we're just inconsciously looking at pupil size.