If the problem statements in lean are correctly formalized, I think we can be pretty confident the proofs are correct, which does not mean their natural language counterparts are faithful representation of the proofs though.
I'm however puzzled by the number of proof claims without lean proofs. How does OpenAI have confidence in those, especially if, as noted in the blog posts, the papers are very hard to read?
Math problems are often stated in a way that makes it possible to automatically verify if a solution is correct. Which means a loop that speculates an approach (LLM and/or prompts), implements it (LLM), then checks (automated) can work. You still need to have a very good LLM, and probably very good prompts with interesting research directions otherwise you can probably loop forever.
> Anyone taking a single look at the ARC-AGI "challenges" can see things a 5 year old could reasonably solve.
Isn't that the goal of these challenges? Each release shows challenges that are very easy for humans, but are impossible for the models at the time of release (which demonstrates some missing generality).
I think I've read the challenge authors say that, the day they cannot make a new challenge, then models are AGI.
Since global warming I guess. When there's a heat wave, Norway is probably one of the worst places on hearth to be. You've got near constant sunlight, there's not much time at night to cool your house.
The matrix multiplication is only deterministic for sparse-dense products under these settings:
> torch.bmm() when called on sparse-dense CUDA tensors
And it's not listed under the operations that raise an exception otherwise, so I'm not sure the docs promise that dense-dense matrix-matrix products are deterministic.
It may be an implementation detail, but in practice, if the only way to get a deterministic output is to run on the CPU, then it's not going to be usable.
Chaitin's constant does not count? Depends on your definition of constructed, but contrary to "easy" normal numbers such as Champernowne's constant, it's not defined by its sequence of digits.
I thought fair use was decided on a case by case basis, and could not be guaranteed? If true, wouldn't that mean that in other cases it could be ruled differently?
> For example, in a variant of environment TR87, Opus 4.6 scores 0.0% with no harness and 97.1% with the Duke harness (12), yet in environment BP35, Opus 4.6 scores 0.0% under both configuration
This is with a harness that has been designed to tackle "a small set of public environments: ls20, ft09, and vc33" (of the arc-agi-3 challenge), yet it looks like it does not solve the full arc-agi-3 benchmark, just some of them.
No, APIs fall under copyright, but the Supreme Court found that Google's reimplementation of Java's API was falling under fair use. Fair use is decided case by case, one cannot use that decision as a precedent.
Not quite in my opinion. The output of an LLM from a simple prompt falls into the public domain, but if you also give a copyrighted work as input, the mechanistic transformation performed will not alter the original license (same as encoding a video does not change its license).
There's a difference between "I've read a LGPL code once, maybe I could do something similar" and "I've been reading this LGPL code for 12 years and now I'm going to do exactly the same thing".
Everyone writes as if he just fed the spec and tests to Claude Code. Ignoring for now that the tests are under LGPL as well, the commit history shows that this has been done with two weeks of steering Claude Code towards the desired output. At every one of these interactions, the maintainer used his deep knowledge of the chardet codebase to steer Claude.
Google vs Oracle ruled that APIs fall under copyright (the contrary was thought before). However, it was ruled that, in that specific case, fair use applied, because of interoperability concerns. That's the important part of this case: fair use is never automatic, it is assessed case by case.
Regarding chardet, I'm not sure "I wanted to circumvent the license" is a good way to argue fair use.
The test suite was also licensed under the LGPL. The reimplementation can be seen as a derivative work of the test suite, and thus should fall under the LGPL. This does not even mention the fact that the coding agent, AND the user steering it, both had ample exposure to chardet's source code, making it hard to argue that the reimplementation is a new ship.
Even the most permissive open source licenses such as MIT require attribution. Releasing as open source would therefore benefit the author through publicity. Bein able to say that you're the author of library X, used by megacorp Y with great success, is a good selling point in a job interview.
I'm in strong agreement with this. Even though I'd prefer winter time all year round, I would rather enjoy permanent summer time over switching twice a year. And I'm living in France, so my "winter time" is actually already a DST compared to the standard time (France's timezone should be UTC like the UK, but WW2 changed that to UTC+1 and we never switched back), so the "summer time" is actually a "double DST".
Not running, but in cycling we have power meters, and some workouts (eg 2 x 20' threshold) will definitely burn in the range of 800 calories in an hour. The energy measured by the power meter for this workout is 800 kJ for me (my threshold being around 260W). Now it turns out the conversion factor from kJ to calories is 1/4, but the body is only 25% efficient when producing calories for cycling, meaning one has to burn 4x the amount measured by the power meter. So that's 800 calories for this kind of workout, for me. I wouldn't be surprised if runners of similar fitness doing similar workouts had the same energy expenditure.
I' m far from being an LLM enthusiast, but this is probably the right use case for this technology: conjectures which are hard to find, but then the proof can be checked with automated theorem provers. Isn't it what AlphaProof does by the way?
The idea is interesting, but I don't think this qualifies as a second factor, as it can be reduced to a factor you have to remember, so equivalent to a password. The second factor should be derived either from something you own, or something that can be obtained from biometry.