gwern

> One of the girls quoted said she loves writing, but finds it helpful to get structure and initial content from generative AI.

Which shows she's not learning from the AI or even improving on it. All she's doing is filling out the 'structure and initial content' - oh, is that all???

> The kid at Columbia has already launched two startups.

Which doesn't require learning anything much from the LLMs, as he did that on his own before them (note the Leetcode). Also, considering that the second one is fraudulent and if it works, exists largely to abet fraud, this is not a ringing endorsement.

> With the advent of new technology, humanity has no choice but to move up the value chain -- in this case, from executing tasks to judging, supervising, and curating AI work.

ChatGPT couldn't written that ending any better. (I will charitably assume it did not, unless you took the trouble to replace that emdash with a double hyphen just to be cute.) It could have written it about 1000x cheaper, however.

Wishful thinking if you read the article. Most of these kids aren't learning anything about how to use AI. They aren't like a Simon Willison or Janus or anything. They're not learning about confabulations, they're not learning about scaling laws or tokenization, they're not learning to use the API, they're not running multiple models to get a feel for how they think differently, they ain't doing none of that shit you wishfully think they are - because they're cheating to minimize effort so they can go do something else like watch Tiktok. (Even Lee, who is by far the most successful and sophisticated user profiled in OP, is just dumping out stuff as fast as he can to try to make a buck.)

They are copy-pasting the assignment from Blackboard to ChatGPT and vice-versa. As the article points out, they usually aren't even reading the outputs to check that the essay doesn't suddenly start bloviating about Aristotle.

And no one needs to pay a college wage premium, or any wage, for that:

> He worries about the long-term consequences of passively allowing 18-year-olds to decide whether to actively engage with their assignments. Would it accelerate the widening soft-skills gap in the workplace? If students rely on AI for their education, what skills would they even bring to the workplace? Lakshya Jain, a computer-science lecturer at the University of California, Berkeley, has been using those questions in an attempt to reason with his students. “If you’re handing in AI work,” he tells them, “you’re not actually anything different than a human assistant to an artificial-intelligence engine, and that makes you very easily replaceable. Why would anyone keep you around?” That’s not theoretical: The COO of a tech research firm recently asked Jain why he needed programmers any longer.

> Charles Eames’s distinction between forced compromises and willingly accepted constraints reveals a profound truth — in the delicate balance between structure and freedom lies the sweet spot where creativity thrives.

Suggested constraint: "write this post without AI".

That would be a good baseline. Maybe the ML part would be for ranking them? Because I expect you would drown in matches, and the hard part becomes finding the good ones.

My suggestion would be that, since you want a tiny integer-only NN tailored for a specific computer, are only occasionally training one for a specific task, and you have a simulator to generate unlimited data, you simply do random search or an evolutionary method like CMA-ES.

They are easy to understand and code up by hand in a few lines (which is one reason you won't find any libraries for them - they are the 'leftpad' or 'isEven' of NNs, the effort it would take to install and understand and use a library often exceeds what it would take to just write it yourself), will handle any NN topology or numeric type you can invent, and will train very fast in this scenario.

Can it be used to covertly run computations invisible to the OS or CPU?

OP fails to link the patent, which is at https://patents.google.com/patent/US748284A/en https://patentimages.storage.googleapis.com/8f/2f/9b/bc599b6...

(The patent is only 2 pages, and the 1pg text description doesn't address the question of anaerobic bacterial decay or the removal of water to solve decay that way, but it also only claims 'indefinite' preservation, so it's not necessarily claiming perpetual preservation. It could just be intended for a few months or years, and who knows how well it'd work?)

> All these methods of sealing the body into airtight contraptions ignores the fact that decomposition comes from within. When the heart stops, the autolysis, or self-digestion, of the flesh begins, through its enzymes consuming cell membranes. Trapping a body in with its own bacteria means it is producing gas that has nowhere to go.

One thing worth noting here is that this was the era of germ theory and vitamin-ization and initial success in blood/organ transplants (eg. https://www.amazon.com/Immortalists-Charles-Lindbergh-Alexis... ). There had been such huge gains in health that some of the projections and assumptions are hard to recognize (and are part of why science fiction from before the 1950s can be so strange to read). For example, given how pervasive germs turned out to be, and how deadly, but also how beneficial things like vaccination or pasteurization were, there was something of a vogue for imagining a future germ-free world: https://gwern.net/doc/genetics/microbiome/germ-free/2012-kir... This would help lead to greatly reduced sickness and mortality, and help extend lifespans to centuries. (After all, the contemporary fad of 'microbiome'-everything aside, mammals like mice or humans get along pretty well without any microbes, as we now know from the later germ-free animal research and bubble boys.)

You might ask how they imagined this working in practice, as sterilizing the entire world seems infeasible? Well, one idea was... Perfectly-sealed glass cubes, using similar techniques as germ-free animal or bubble boy environments. For example, the 1927 "The Machine Man of Ardathia" https://freeread.com.au/@RGLibrary/FrancisFlagg/Ardathia/The... imagines a far future germ-free man traveling back in time in his glass cube to inspect us short-lived disease-plagued primitives who wander around naked. So it's not a big step to imagine sealing them into a block for indefinite preservation post-mortem.

An interesting ML exercise (possible class project!) would be to try to automate this. A bigram corpus combined with word embeddings and a NSFW text classifier, maybe? Your usual word embedding might not work because the point is the multiple meanings, so maybe the twist would be that you need a polysemous word embedding with multiple vectors or something like that, so it's not just an off-the-shelf word2vec...

LLMs are also heavily biased after chatbot tuning leads to mode-collapse. That's why you see the same verbal tics coming out of them, like the em-dashes or the 'twist ending' in the more recent 4os. And if LLMs really were unbiased, you'd expect better scaling when you tried to bruteforce code correctness. Training a 'test LLM' will just wind up inheriting a lot of the shared blindspots. They aren't independent of the implementation at all (just like humans are not independent, even when they didn't write the original, and didn't see it either; and this is why you can't simply throw _n_ programmers at a piece of code and be certain you got all the bugs, and why fuzzers will continue to rampage through code).

My point is that there are no 'genuine consequences'. We are told that supposedly there is selection and that the predictive power is nearly perfect and everyone relies on solely genetics to do everything; and yet, we do not see a world which reflects such effects at all. This worldbuilding does not make sense from a scientific perspective. Instead, what we see is a world in which you have extremely cheap sequencing in a panopticon and a caste society based on seemingly-random arbitrary discrimination with no actual consequences to any supposed inferiority or superiority*, akin to the classification of supposedly-objective 'classes' in North Korea where everyone is finely graded on hereditary loyalty to the elites but could be knocked down a rung at any time for suspected disloyalty.

(My headcanon is that the esoteric story of _Gattaca_ is that none of the supposed embryo selection or valid/invalid screens exist at all, any embryo selection is just ordinary IVF quality control for aneuploidy etc, there is only cheap universal sequencing developed by a totalitarian police state like the CCP (see: Zero COVID testing), and the classification exists to manufacture distinctions and divide families and punish/reward elite supporters; the Freemans were some lower 'wavering' class, grouped with religious minorities, as indicated by their dissenting 'natural' birth, and so they earned a valid vs invalid split within their family. It makes more sense than the actual story, anyway.)

* I've remarked elsewhere that _Gattaca_ is an example of how movies with a message tend to fail by loading the dice too heavily for one side, removing any 'moral dilemma', and it is no exception: we never see any evidence that Vincent is doing anything wrong or that he shouldn't go on the space mission. In fact, given his methodical penetration, high competency, motivation, and extraordinary success at getting into the mission, he shows that he should be selected for his moxie and chutzpah. So what _Gattaca_ should have done was to make the ending Vincent suddenly clutching his heart from the shock, and fade to black.

I read this the other day and did a little more reading about his _Autobiography_ project, and Twain's early life was even harsher than I had realized: https://gwern.net/doc/psychiatry/bipolar/energy/2023-cavitch...

At that sort of tariff, you'll see a lot of people dabbling in tariff evasion by smuggling stuff in their luggage (whether they realize it or not). If you pull it out of the packaging and just put it in your checked-in baggage, I doubt US Customs would know what it was or that you didn't bring it with you. "I'm a digital nomad, so I like to bring a lightweight monitor to work on."

Some of the older posts are very cute. I enjoyed https://greydanus.github.io/2020/12/01/scaling-down/ on showing how many high-powered tricks you can do with a tiny NN of the sort that will train in seconds.

I don't know if I'd recommend it, but checking, Sawyer's _Group Genius: The Creative Power of Collaboration_ book hits most of the points I'd cite about brainstorming in its chapter, so you could always start there.

I enjoy reading obituaries, for a number of reasons. (Mostly in the NYT, but also, of course, the famous Economist obits.) But I rarely have ever gotten any creative ideas from them, and I do not recall hearing about anyone else doing so either.

And I object in principle to telling silly stories about desirable activities to try to justify them. This is how we get bad ideas like 'Mozart for babies' or 'we should make kids play chess because it correlates with being smarter'. Chess should be played for its intrinsic value, because if it can only be justified by its instrumental value in raising IQ or grades, the case for that was weak, much weaker than its proponents were willing to admit, and has since turned out to be false - but along the way, wasted a lot of time and money and made a lot of kids miserable playing a game they didn't like, and the proponents just made the world a worse place by wasting all those resources, filling the literature with useless analyses and research on chess correlates, and decreasing public trust in science.

None. But then, that's still more than OP was able to adduce, as they provided no examples at all, neither groundbreaking nor creative. Which is my point: the bar of 'an anecdote' is so very low. You can provide a anecdote for the creative value of smelling rotten apples in a drawer (Schiller); you can provide an anecdote for the creative value of drinking 50 coffees a day (Balzac); you can provide an anecdote for countless crazy things people have done! But you can't provide an anecdote about reading obituaries?

It might be doable soon, you're right. But there seems to be a substantial weakness in vision-language-models where they have a bad time with anything involving screenshots, tables, schematics, or visualizations, compared to real-world photographs. (This is also, I'd guess, partially why Claude/Gemini do so badly on Pokemon screenshots without a lot of hand-engineering. Abstract pixel art in a structured UI may be a sort of worst-case scenario for whatever it is they do.) So that makes it hard to do any kind of feedback, never mind letting them try to code interactive visualization stuff autonomously.

I'm not the one claiming anything specific about robotaxi economics. What has been asserted without evidence can be denied without evidence. And I can think of plenty of reasons that there could be kinks or large elasticity changes: for example, thresholds where it becomes cheaper than owning a car in many places, which is currently not true anywhere in the USA AFAIK.

The quality would not follow because Distill.pub publications take literally hundreds of man-hours for the Distill part. Highly skilled man-hours too, to make any of this work on the Web reliably. (Source: I once asked Olah basically the same question: "How many man-hours of work by Google-tier web developers could those visualizations possibly be, Michael? 10?")

> Now, if the cost was 35% less per ride than it is today, how much bigger would the market be? 1.5×? It is certainly not much more.

"Source?" "I made it up."

There's lots of legitimate research related to creativity which could be discussed: the existence of the 'incubation effect' and sleep effects, the inefficacy of the popular forms of 'brainstorming' compared to the more painful forms that work*, the 'near adjacent', the 'equal-odds rule', the benefits of cognitive/ideological diversity (and lack of benefits of certain other kinds of diversity), the correlation with intelligence and personality traits like Openness...

As opposed to OP. Which adduces so little evidence for the claim about reading obituaries that a rando like me could actually write a more persuasive argument for the benefits of reading obituaries (because I at least wrote one thing tenuously inspired by reading an obituary the other month: https://gwern.net/traffic-lights ).

Even the most shameless periodical usually tries for at least 3 anecdotes, no matter how dubious and strained, before declaring it the hot new trend or It Is Known fact.

* One of Sawyer's research topics, as it happens.

> It’s not just about learning new facts, of course — it’s about asking questions. Why was a British mystic in Mexico City? How did Spanish-language television evolve in the U.S.? What led someone to invent PLAX or build search tools for financial news decades before Google? Even if you don’t find all the answers, just posing the questions helps you flex the creative muscle that thrives on curiosity and connection.

Maybe wait until you have at least 1 anecdote, anywhere in the history of the world, of major creativity from reading an obituary, before recommending it?

This always frustrates me about paper abstracts. Whether it's economics or AI, everyone seems to make a point of being as vague as possible, when it would sometimes take 1 tiny word to clarify it hugely . You'll see paper abstracts talk about how they analyze "an important and widely deployed commercial family of large language models (LLMs)" and then you have to skim 10 pages before you finally find out that they mean 'GPT'.

I don't think the authors are even doing it maliciously or deliberately, because it's like how students or kids struggle to write anything. It's just a fallback when you're struggling to condense it and have gotten lost in your forest. Like how you can ask someone, "OK, that's all great, but what did you do? What are you trying to say here?" "Oh, I Xed the Y with Z." "There you go. That's your abstract."

Nope! The knowledge cutoff does not show lack of leakage. Even if you get a non-confabulated cutoff which was before anyone ever asked the strawberry question or any question like it (tokenization 'gotchas' go back to at least davinci in June 2020), there is still leakage from the RLHF and tuning process which collectively constitute post-training, and which would teach the LLMs how to solve the strawberry problem. People are pretty sure about this: the LLMs are way too good at guessing things like who won Oscars or Presidential elections. This leakage is strongest for the most popular questions... which of course the strawberry question would be, as it keeps going viral and has become the deboooonkers' favorite LLM gotcha.

(This is, by the way, why you can't believe any LLM paper about 'forecasting' where they are just doing backtesting, and didn't actually hold out future events. Because there are way too many forms of leakage at this point. This logic may have worked for davinci-001 and davinci-002, or a model whose checkpoints you downloaded yourself, but not for any of the big APIs like GPT or Claude or Gemini...)

Probably not, as economic things like living standards or quality of life generally seem to be logarithmic and have diminishing returns, but why not make a list and show us?

I guess I misremembered all those playdates I had as a kid in the 90s, where the other kids didn't live close enough to simply walk or bike to (which, as a '90s kid, I was still able to do, although American parents apparently slammed the window firmly shut on that starting in the '00s), and we had to rope the adults in for car transport. After all, varjag knows better than I do, and knows they were actually in the 1980s.

I hadn't thought about mangoes but now that you mention it, it does feel like they showed up in the USA at some point since the 1990s.

> - sugar free sodas

I am not convinced this was actually a good thing. After all these decades of studies in ever more obese human populations, it should not be this hard to find any benefits at all, and there should not be so many warning signs. You certainly can't see it in any obesity rates or soda tax results (or my relatives...), the experimental weight loss results have always been oddly unimpressive, and there are lots of concerning results about the effects on microbiomes (https://gwern.net/doc/genetics/microbiome/2022-suez.pdf) or on cardiovascular health (https://gwern.net/doc/biology/2024-witkowski.pdf https://pmc.ncbi.nlm.nih.gov/articles/PMC10334259/), which is all very weird if they are 0 calories and don't affect insulin metabolism, as you would expect unambiguous, immediate, large benefits (contrast this with, say, GIP/G1P). So, maybe sugar-free will finally wind up being a wash or slightly beneficial, but it's not going into my list either way.

(Also, I eat mangoes and frozen vegetables and a little reluctantly eat nü brussel sprouts, but I don't drink soda, so it's against the spirit of my personal list to begin with.)

As an American, I do indeed have little idea what sorts of improvements people in the EU as a whole - much less any individual country - have had. I can guess at a few, since presumably a lot of my items transfer. (Pascalization for avocado preservation, for example, is presumably a world-wide thing because the laws of physics are the same everywhere.) But I can't write a decent list.

I see that fact as more of an invitation to write your own. Do Europeans feel annoyed at the omissions of XYZ? Go right ahead and make your own! There's no patent on the idea of 'writing down a list of things that have gotten better'. I'm not going to sue you if you write down a list of ordinary things that have gotten nicer in the UK. In fact, I'd be happy to link it in the external links section if anyone makes a decent list for some other country in the same spirit. (I would prefer it not be tech-heavy, though. Things like "did you know PCs and smartphones also got really cheap in the Third World or the EU, and SSDs are pretty sweet compared to HDDs?" are not particularly interesting to anyone and do not need to be pointed out.)

> How many examples does OpenAI train on now that are just variants of counting the Rs in strawberry?

Well, that's easy: zero.

Because even a single training example would 'solved' it by memorizing the simple easy answer within weeks of 'strawberry' first going viral , which was like a year and a half ago at this point - and dozens of minor and major model upgrades since. And yet, the strawberry example kept working for most (all?) of that time.

So you can tell that if anything, OA probably put in extra work to filter all those variants out of the training data...

Yeah, early 2000s lines up with what I can tell. There were a few translations like https://web.archive.org/web/20021207080806/http://www.berdya... which surfaced somewhere around 2001 or 2002, and they appear to have been about the first anyone had ever heard of 'Cosmism'. Which still wasn't much at all - I was on SL4 myself starting around 2003, and I don't remember ever hearing of it (except possibly as a footnote to a footnote on Konstantin Tsiolkovsky) until Stross's post. So you can imagine my surprise to learn from Stross that despite me never having heard the name, and having never seen any of this discussed on SL4 or Extropians or OB or in _Great Mambo Chicken_ or in Ettinger's _The Prospect of Immortality_ and Yudkowsky and Metzger and Max More and all the rest of us having never heard anything about this, this was nevertheless foundational to Western transhumanism... Remarkable the things you learn if you hang around long enough!

Sarcasm aside, that was a bit of a pity, because even if it had no relationship to anything in 'TESCREAL', Cosmism is an interesting historical artifact. When I was in LA back in 2019, I was able to visit the Museum of Jurassic History where there was an exhibit of Tsiolkovsky stuff like drawings on how humanity might live in space, and it was much more interesting when you knew a little bit about the Cosmism background there.