3,775 karma · joined June 17, 2021
For doctors an aggregate 1% chance is low, but the realization of the outcome in a single person is not a percentage, its binary bad side effects or not.
Admittedly, doctors could usually do better in communicating risks (e.g. see the famous example of 1/3 risk of impotence, which patients interpret either as 1 out of 3 people, 1 out of 3 times having sex, or 1/3 of the time of sex).
"“The study cannot contribute to the recent debate on whether or not social-media use is harmful, or whether or not smartphones should be banned at schools,” because the study was not designed to answer these questions, says Tobias Dienlin, who studies how social media affects well-being at the University of Vienna. “Different channels and uses of the Internet have vastly different effects on well-being outcomes,” he says."
My concern is an ecological one within science, and your argument addresses the frontier of scientific methods.
I am sure both are compatible. One interesting question is what instruments are suitable to reduce negative externalities from bad actors. Pre-registration works, but is limited to few fields where the stakes are high. We will probably similarly see a staggered approach with more restrictive methods in some fields and less restrictive ones in others.
That said, there remain many problems to think about: E.g. what happens to meta-analyses if the majority of findings comes from the same mechanism? Will humans be able to resist the pull of easy AI suggestions and instead think hard where they should? Are there sensible mechanisms for enforcing transparency? Will these trends bring us back to a world in which trust was only based on prestige of known names?
Interesting times, certainly.
I assume you mean that LLMs can generate new insights in the sense of producing plausible results from new data or in the sense of producing plausible but previously unknown results from old data.
Both these things are definitely possible, but they are not necessarily (and in fact often not) good science.
Insights in science are not rare. There are trillions of plausible insights, and all can be backed by data. The real problem is the reverse: Finding a meaningful and useful finding in a sea of billion other ones.
LLMs learn from past data, and that means they will have more support for "boring", i.e. conventional hypotheses, which have precedent in training material. So I assume that while they can come up with novel hypotheses and results, these results will probably tend to conform to a (statistically defined) paradigm of past findings.
When they produce novel hypotheses or findings, it is unlikely that they will create genuinely meaningful AND true insights. Because if you randomly generate new ideas, almost all of them are wrong (see the papers I linked).
So in essence, LLMs should have a hard time doing real science, because real science is the complex task of finding unlikely, true, and interesting things.
With regard to the debate, I think it's good not to engage in too much black-and-white thinking. Science itself is a pretty muddy affair, and we still haven't grown beyond simplistic null hypothesis significance testing (NHST), even decades after its problematic implications became clear.
That's why it's so important to look at the macro implications: I.e. how does this shift costs? As another comment nicely put it, LLMs are empowering good science, but they are potentially empowering bad science at an order of magnitude more.
thanks for the honest and thoughtful discussion you are conducting here. Comments tend to be simplistic and it's great to see that you raise the bar by addressing criticism and questions in earnest!
That said, I think the fundamental problem of such tools is unsolvable: Out of all possible analytical designs, they create boring existing results at best, and wrong results (i.e. missing confounders, misunderstanding context ...) as the worst outcome. They also pollute science with harmful findings that lack meaning in the context of a field.
These issues have been well-known for about ten years and are explained excellently e.g in papers such as [1].
There is really one way to guard against bad science today, and that is true pre-registration. And that is something which LLMs fundamentally cannot do.
So while tools such as data-to-paper may be helpful, they can only be so in the context of pre-registered hypotheses where they follow a path pre-defined by humans before collecting data.
[1] http://www.stat.columbia.edu/~gelman/research/unpublished/p_...
First of all, it's inevitable that LLMs will be/are used in this way and it's great to see development and discussion in the open! That's really important.
Secondly, this will absolutely destroy some areas of science even more than they have already been.
Why? First, science as all of humankind is always a balance between benevolent and malevolent actors. Science already battles data forgery, p-hacking and replication issues. Giving researchers access to tools like this will mean that some conventional quality assurance processes will fail hard. Double-blind peer review will no longer work when there are 10:1 or 100:1 AI generated to high-quality submissions.
Second, doing analysis and writing a paper is one bottleneck of science, but epistemologically, it's not the important one. There are innumerable ways to analyze extant data and it's completely moot to do any analysis in this way. Simmons, Nelson and Simonsohn / Gelman et al. etc have shown: Given a dataset, (1) the findings you can get are practically always from very negative effects to very positive effects, depending on the setup of the analysis. So having one analysis is pointless, especially without theory. (2) even when you give really good labs the same data and question, almost nobody will get the same result (many labs experiment).
What does this tell us? There are a few parts of science that are extremely important and without them science is not only low-impact, it even has a harmful effect by creating costs for pruning and distilling findings. The really important part are causal analyses, and they practically always involve data collection. That's why sciences with strong experimental traditions fare a bit better - when you need to run a costly experiment yourself in order to publish a paper, this creates a strong incentive to think things through and do high-impact research.
So yeah, we've seen this coming and it must create a big backlash that prevents this kind of research from being published, even if vetted humans.
Source: am a scientist, am a journal editor.
ev loops instead are a mid-layer concept that sits below other infrastructure such as threads and processes. And (perhaps somewhat frustratingly) it is not too uncommon to have multiple ev loops in parallel. See for example the proxy.py project, which offers to run one async loop per process for a speedup.
As a result, there are some incentives to swap out the loop itself, e.g. for faster implementations like uvloop, because they are somewhat pluggable anyways.
[edit] Seems that nicotine is not really harmless, even in gum or patches: double the risk of heart palpitations and chest pains, 1.67 the risk of nausea and vomiting, 1.5 times the risk of gastrointestinal complaints, and 1.4 times the risk of insomnia.
Of course, if you measure anything this closely with >100k participants in RCTs, it's bound to have a significant effect. But these odds ratios are not exactly small.
The problem with these is that there are almost always equally plausible theories, and it's really hard to figure out which is correct.
Especially given the huge amount of effort that has gone into research on tobacco and nicotine (I guess millions of hours, if not more), I would definitely NOT say we are too quick to dismiss the possibility.
I'm sure you can find an interesting econometric paper presenting this argument, but then it would be more credible.
Here is one like that: ~500k individuals, vaping has an odds ratio of 1.33 (95% CI = 1.14–1.56) for heart attacks compared to non-smoker/non-vapers.
https://www.sciencedirect.com/science/article/abs/pii/S01675...
[edit] To compare that to caffeine, this study does not see a significant risk increase for coffee drinkers. So it seems that vaping is objectively more harmful:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5940396/
[edit2] And an umbrella review, because I like them a lot: This shows that negative effects of high coffee intake mostly disappear when controlling for smoking. Note this paper has a correction, which I think is a great sign of credibility. https://www.bmj.com/content/359/bmj.j5024
As far as I see, much of European AI regulation tries to contain fallout where existing laws are under threat to be undermined (i.e. authorship, safeguarding public opinion, competition, harm/safety etc.)
So I'm unreasonably excited by bittorrent, storage based on consistent hashing, and rsync (rolling checksums are just amazing).
Interestingly I hate distributed consensus with a passion because it always seems to achieve the opposite - causing more complexity.
My takeaway is that matplotlib may be a good alternative for specific cases where you need much more control.
That said, I think it's a refreshing perspective, and it seems like a good idea to look at effect heterogeneity in psychedelics use. Thinks like bayesian trees (BART) can help identify subgroups that react differently to stimuli along specific characteristics.
Of course, if you use different models, training, fine tuning etc. the benchmarks will differ depending on ram, support of fp8 etc.
- Valuing work gives meaning (money, appreciation)
- Autonomy gives feeling of control
- Managing burden prevents overwork, exhaustion and fatigue
Time to push for a change. And time to call some people and ask whether they are truly ok right now.
- a bayesian mixed model is just what I would have used here, great choice and a good sign for power
- the effect sizes are definitely very small. I would not overstate the story here.
That said, I also don't believe that facebook is the primary story here, because other findings all suggest tiny effect sizes (see the famous emotional contagion experiment) and because we don't have a clear, convincing theoretical model IMO. There are just so many intra-individual confounders that would need to be included, including some that have dynamic, self-reinforcing effects.
Still, great to have more robust work here.
[edit] PS: just to throw a methodological nitpick out: Some bayesians are going to have a heart attack while reading "the uncertainty cutoff of 97.5% for posterior probabilities".