gjulianm

That might have an effect, but these studies are probably mostly selecting for people who can tolerate a hostile environment for longer, which are usually healthier. I find it unlikely that sauna alone explains the fantastic, almost miraculous hazard ratios that these studies report.

> Hammam's temperatures are around 40-50 degrees Celsius and humidity is close to 100%.

Which makes it absolutely unbearable. By the way, that combination of temperature + humidity will cause severe hyperthermia (which can be deadly) faster than people think.

I doubt they would replicate it or any of the magical effects of saunas. Lots of the sauna studies suffer from the same issue where people self-report sauna usage rather than being assigned randomly to a treatment group. In countries where saunas are readily accessible and most people are under the impression that the more you use sauna the healthier you are, the ones that use the sauna less are probably because they tolerate it far worse. And that's probably related with age, comorbidities, physical condition, etc.

Basically, the sauna studies are probably mostly discovering that "healthier people can stand sauna longer". In countries where most people don't stand sauna for more than a few minutes, that self-selection bias won't exist.

> and all of a sudden one of them starts changing in size at a rate disproportionate to its history, that's probably cause to dig a little deeper.

That sentence is doing a lot of heavy lifting.

- What's "disproportionate to its history"? Obviously something going from 1mm to 10cm is worth checking out, but what about something going from 1mm to 2mm? Might be a tumor, might be that the position is just slightly different.

- What about other less measurable factors? Example, border features. That's harder to measure and things like movement or different machines can change how the borders of a feature look. How do you know what's a baseline and what's not.

- How frequently do you run these scans? It's likely that if something "starts changing in size" suddenly it will start giving symptoms before you have your next scheduled scan.

> It's certainly not "throw more data at the problem". Instead it's about giving the data a time axis with some decent fidelity.

It's definitely throwing more data at the problem, and you're assuming that it's viable to give "a time axis with decent fidelity". MRIs are much more complicated to interpret than people think, and screening is a much harder problem too. There are a lot of studies testing MRI imaging as a screening technique (among other techniques) and they don't always show an increase in survival rates.

> we wait until something is clearly wrong to get the MRI. f you want to find an MRI scan of an early stage asymptomatic cancer, the only data on that will be the exceedingly rare case that someone has something else unrelated wrong

Not always. There are bunch of studies for MRI screening in high-risk populations for specific cancers. There are scoring systems for a lot of them based on imaging features and they do find asymptomatic cancers.

In fact, if you add low-risk populations to the studies used to design imaging scores, you might end up adding more noise and making the study more difficult and the scoring less accurate.

And the fact that they have a complaint (or have known risks) makes it easier to classify, compare and understand the data.

> That's actually exactly why it's important to have MRIs more frequently to be able to establish baselines and identify trends as they develop.

How? How do you establish baselines? How do you build a classification of incidental findings? It's very possible that you'll find a lot of types and not a lot of representatives of each type. And then you have to correlate that to actual clinical results, but the population will be so heterogeneous that it'll be really hard to find an actual result.

It's not just "let's throw more data at the problem".

> What this fails entirely to capture is that doing something to increase your odds of survival, damn the consequences, is an individual choice.

What you're failing to capture is that this is a hard problem because it's both an individual choice and a collective one as well. Those "terrible side effects" might actually end up killing someone. You're choosing between a high-chance lottery on a small population or a low chance lottery on a far larger one. It's not that simple.

Unfortunately that "chosen threshold" is really hard to know, specially if you want to balance individual and population level necessities.

See https://pubmed.ncbi.nlm.nih.gov/38926075/. I was not aware of the ERSPC which came out late last year and gives better outcomes for screening, but overall the evidence is not super clear yet. There are possibly certain groups that can benefit from PSA screening more than others. Also, modern, more effective treatments might allow for later diagnosis with the same clinical results.

> Maybe the right answer isn't to do a biopsy, but to monitor the area with follow-up scans?

Doctors have already thought of this. Several issues with it:

* Monitoring still causes anxiety and mental health issues which come with real effects on patient's quality of life. It's not "harmless".

* Unclear when to monitor and when to treat. It's also really hard to get enough data to characterize these early unspecific findings enough to get confidence on what to do.

* Monitoring via MRI might be just as useful as monitoring via symptoms or any other "passive" methods that do not require a previous scan.

> This is much more sensible than just not testing at all and letting people die from cancer.

This is not what happens. You're assuming that if the cancer does not get detected by the screening then it never gets detected. What actually happens is that the test gives information that might actually be redundant and obtainable in less risky way. What the studies are showing is that waiting until there are other, more specific signs and symptoms of the prostate cancer results in the same survival rates.

> If the test detects cancer that doesn't need treatment, don't treat it!!

How do you know which ones to treat and which ones to leave?

> With my change: 95% of people who are shown scans have cancer and are treated earlier. Without my change: many of those 95% die

Why? What happens if the cancer still doesn't respond to treatment even when detected early? Or, to the contrary, if the cancer also responds to treatment when it starts becoming symptomatic?

That's why we have studies to understand if screening is a good practice or not. It's not that clear cut.

> it's likely this result in innovations that would drive down costs, improve accuracy, as well as producing a much larger corpus of data with which to guide diagnosis and reduce false positives.

Why is it likely? We already have a lot of MRI data. There are already a lot of incidental findings. It might also be an issue of the MRI not being able to produce enough information to discriminate.

> To use a software analogy, if your downtime detection system kept producing false negatives, would your solution to be just turn it off? You'd get some better night's sleep, but you'd pay for it when the system really went down and you had no idea.

The analogy is rather something like this: your downtime detector is not just a "ping" but a full web browser that tests everything and it sometimes flags things that are not actually issues. So you don't turn it off, but you only use it when you have another signal that indicates that something might be going wrong.

The argument for better screening would require that finding those asymptomatic cancers actually improves survival rates. There are several reasonable scenarios where early screening doesn't improve it:

* The cancer is aggressive and resistant to treatment. Chemo/radiation only pause the growth for a bit, but ultimately the cancer keeps growing and the total survival time is the same (only that the patient spent more time knowing they had cancer).

* The cancer is susceptible enough to treatment that it's still curable when it becomes symptomatic and found through other means.

* The cancer is slow enough that the patient dies from other causes before.

Early screening brings benefits only when the cancer ends up causing issues and responds differently to treatment between the "early screening detection" time and the "normal detection" time.

It's impossible to know beforehand which of the scenarios have more weight, specially because we have very little data on what happens way before cancer is detected via the usual methods. We need better studies on this, and for now the evidence doesn't really point out to these large, indiscriminate screenings being actually helpful.

> Swap out "AI" for any other group and see how that sounds.

- AIs should not take issues that are designed to onboard first time contributors - Experienced matplotlib mantainers should not take issues that are designed to onboard first time contributors

Sounds about the same

Honestly, I've been using the frontier models and I'm not sure where people are seeing these massive improvements. It's not that they're bad, it's just that I don't see that much of an improvement the last 6 months. They're so inconsistent that it's hard to have a clear idea of what's happening. I usually switch between models and I don't see either those massive differences either. Not to mention that sometimes models regress in certain aspects (e.g., I've seen later models that tend to "think" more and end up at the same result but taking far more time and tokens).

> The goal is to assess the current capabilities of a largely autonomous software-building pipeline: it's not yet limitless, but better than it was. What a shocker.

Of course, but we're trying to assess the capabilities by looking at the LLM output as if it were a program written by a person. If someone told me to check out their new C compiler that can build the kernel, I'd assume that other basic things, such as not compiling incorrect programs, are already pretty much covered. But with an LLM we can't assume that. We need to really check what's happening and not trust the agent's word for it.

And the reason why it's important it's because we really need to check whether it's actually "better than it was" or just "doing things incorrectly for longer". Let's say your goal was writing a gcc replacement. Does this autonomous pipeline get you closer? Or does it just get you farther away through the wrong path? Considering that it's full of bugs and incomplete implementations and cannot be changed without things breaking down, I'd say it seems to be the latter.

> This was the equivalent of a "weekend project", and it's amazing

I mean, $20k in tokens, plus the supervision by the author to keep things running, plus the number of people that got involved according to the article... doesn't look like "a weekend project".

> Building a C compiler which can correctly compile (maybe not link) the modern linux kernel is damn hard.

Is it correctly compiling it? Several people have pointed out that the compiler will not emit errors for clearly invalid code. What code is it actually generating?

> Building a C compiler which can correctly compile sqlite and pass the test suite at any speed is damn hard.

It's even harder to have a C compiler that can correctly compile SQLite and pass the test suite but then the SQLite binary itself fails to execute certain queries (see https://github.com/anthropics/claudes-c-compiler/issues/74).

> which, in comparison with a correct modern C compiler, is far less performance critical, complex, broad, etc.

That code might be less complex for us, but more complex for an LLM if it has to deal with lots of domain-specific context and without a test suite that has been developed for 40 years.

Also, if the end result of the LLM has the same problem that Anthropic concedes here, which is that the project is so fragile that bug fixes or improvements are really hard/almost impossible, that still matters.

> it really seems that the complaints here aren't about the LLMs themselves, or the agents, but about what people/organizations do with them, which is then a complaint about people, but not the technology

It's a discussion about what the LLMs can actually do and how people represent those achievements. We're point out that LLMs, without human supervision, generate bad code, code that's hard to change, with modifications specifically made to address failing tests without challenging the underlying assumptions, code that's inconsistent and hard to understand even for the LLMs.

But some people are taking whatever the LLM outputs at face value, and then claiming some capabilities of the models that are not really there. They're still not viable for using without human supervision, and because the AI labs are focusing on synthetic benchmarks, they're creating models that are better at pushing through crappy code to achieve a goal.

> Read the freaking article

The freaking article omits several issues in the "compiler". My bet is because they didn't actually challenged the output of the LLM, as it usually happens.

If you go to the repository, you'll find fun things, like the fact that it cannot compile a bunch of popular projects, and that it compiles others but the code doesn't pass the tests. It's a bit surprising, specially when they don't explain why those failures exist (are they missing support for some extensions? any feature they lack?)

It gets less surprising, though, when you start to see that the compiler doesn't actually do any type checking, for example. It allows dereferences to non-pointers. It allows calling functions with the wrong number of arguments.

There's also this fantastic part of the article where they explain that the LLM got the code to a point where any change or bug fix breaks a lot of the existing tests, and that further progress is not possible.

Then the fact that this article points out that the kernel doesn't actually link. How did they "boot it"? It might very well be possible that it crashed soon after boot and wasn't actually usable.

So, as usual, the problem here is that a lot of people look at LLM outputs and trust what they're saying they achieved.

The goalpost is not moving. The issue is that AI generates code that kinda looks ok but usually has deep issues, specially the more complex the code is. And that's not being really improved.

> What I want to see is mathematicians employ the same rigor of journalists using abbreviations: define (numerically) your notation, or terminology, the first time you use it, then feel free to use it as notation or jargon for the remainder of the paper.

They already do this. That is how we all learn notation. Not sure what you mean by numerically though, a lot of concepts cannot be defined numerically.

> why do u think I am capable of memorizing the Greek one

No one memorizes the Greek alphabet. We just learn it as we go because it’s useful to have different types of letters to refer to different types of objects. That’s it.

> I tried to get into set theory thrice now, not happening with all the math lingo, hieroglyphs and dry ass content.

That sounds like you’re trying to learn a specific field without actually having any of the prerequisites to learn it. I don’t know what you’re specifically referring to when you say “set theory” as that’s an incredibly wide field, and depending on what you’re trying to learn it can be quite technical.

> Learning can be incredibly fun if it was designed fun. Math is a dry and slow process.

This sounds like someone complaining that getting to run a marathon is tiresome and hard. Yes, teaching mathematics can always be improved and nothing is perfect, but it will still be hard work.

Again, learning notation is part of the process of learning math. No one is gatekeeping anything, at no point you need to do an exam or magically be aware of notation that you never saw. Every book and every class will define new notation at the beginning, in most cases they will do so even when there’s no new notation. I am not sure what your argument is.

I do not have any confusion with the notation, I am confused about what the argument you’re trying to convey with English words.

The only thing that sentence says is that it’s impossible to understand math without understanding the language of math and how it is constructed. Not sure how that is controversial or gatekeeping. If you are annoyed at that comment saying “learn” instead of “be taught”, I think that’s a pedantic argument because the argument wasn’t about that at all.

I know it is a matrix, the notation is not confusing at all. I am saying that the concept of a matrix as a set of numbers arranged in a rectangles and the concept of operations on a matrix are very different things, the confusion will not come from notation.

Well, obviously they will be confused because you jumped from a square of numbers to a bunch of operations. They’d be equally confused if you presented those operations numerically. I am not sure what it is you want to prove with that example. I am also not sure that a child can actually understand what a matrix is if you just show them some numbers (i.e., will they actually understand that a matrix is a linear transformer of vectors and the properties it has just by showing them some numbers?)

The article does not complain about notation. It describes how the different fields of mathematics are so deep and so abstract that it’s hard to understand them as a professional mathematician in a different field. That’s a hard problem worthy of discussion, but as the article says, it’s not as much a problem of notation or of explanations, rather than it’s just intrinsically difficult and complex because these are abstract and deep fields.

It’s not gatekeeping. It’s just hard.