holy crap, this is so good. How did it get buried?
733 karma · joined November 26, 2022
Wow. This is the most insightful statement I've read today.
It's finally clicked [U+2014] why I prefer e-ink to hardback, in spite of the alluring texture of plup across fingers.
It's because it should've been white-on-black, not black-on-white. It makes so much intuitive sense.
- the common sentiment of "child raising is too expensive"
- the reality that wealth has drastically gone up
I think: okay, it must feel expensive for some reason. Probably because the work involved, despite not changing too much in an absolute sense, is relatively much pricier compared to all modern cheap sources of happiness.
Then, this notion of the cost-of-fun is easily transferred to general socialization & the loneliness problem.
The 'loneliness epidemic' is merely the result of weakening demand, owing to a slew of low-cost alternatives. Thus, we end up with two options,
1. automate the social experience
2. accept that the comparative cost of socialization will grow higher forever
For some reason, the vast majority of humans in the 21st century are interested in morally rejecting (1), thus ensuring (2) as an outcome.
.
Note: this is not to say I reject the notion that individuals can be helped. I think most comments in this thread are quite healthy, even as they narrowly focus on the individual case.
But it is rather impractical to adopt a positivist "how you can help" framing to address the epidemic at large. While certainly instrumentally useful, it is necessarily unlikely for the same traditional solutions to loneliness to spontaneously 'gain influence' against what has thus far been a gradual decline in their effectiveness and buying power.
Still clearly effortful work, though. I don't want to disparage it.
Continually sad to see each day that the vast majority of users cannot classify
xai's only failure was to implement this modicum of damage control against social exposure
So, for your quote, a skeptical interpretation of the text may assert the author was merely praising SV in the same fashion one might appraise the party.
great line
And the paper is clearly the ancestor to the article itself, based on the date (5dec -> 11dec)
The author doesn't grasp how much of what they've written amounts to flexing their own outlier intelligence; they must sincerely believe the average programmer is capable of juggling a complex 500 line program in their heads.
I will be running the 120B on my 2x4090-48GB, though.
I was told by an employee that GDM internally has a credits system for TPU allocation, with which researchers have to budget out their compute usage. I may have completely misunderstood what they were describing, though.
MFU can certainly be improved beyond 40%, as I mention. But on the point of small models specifically: the paper uses FSDP for all models, and I believe a rigorous experiment should not vary sharding strategy due to numerical differences. FSDP2 on small models will be slow even with compilation.
The paper does not tie embeddings, as stated. The readout layer does lead to 6DV because it is a linear layer of D*V, which takes 2x for a forward and 4x for a backward. I would appreciate it if you could limit your comments to factual errors in the post.
if they're not too boring, and you're not doing anything important, you'll read it for fun
The non-deterministic outputs are really similar, yeah, if you check the gist examples I linked https://gist.github.com/152334H/047827ad3740627f4d37826c867a.... This part is at least no surprise, since the randomness should be bounded.
I suspect OpenAI will figure out some way to reduce the randomness at some point, though, given their public commitment to eventually adding logprobs back to ChatCompletions.
Perhaps the writer kept them unmentioned in an attempt to reduce demand?
And that's how you end up with CS students aceing theory exams, while completely flunking the coding exam...
> Computers are not at all human, in that they do exactly what someone has set them up to do, which is often not what they thought they were doing, while many beginners expect them to "understand what they meant" and act accordingly. Every simple-looking capability is burdened with detail: the computer "knows what time it is" (thanks to some nontrivial engineering with some possible failure points); the out-of-order CPU "runs just like an abstract in-order machine, but very fast" (until security researchers find a difference); DNS "resolves domain names to IPs" (but is frequently intercepted by networks, and can also serve as a covert backchannel); video codecs "make videos smaller" (but are also complex domain-specific programming languages); text rendering "is just copying bitmaps into the right places" (unless you care about Unicode or antialiasing or kerning).
Although people don't really encounter complex leaky abstractions like that as a beginner in coding. More likely, they'll encounter some simpler poor abstraction like Scratch blocks not being all fully composable in an intuitive fashion, or a sheer wall of complexity (e.g. getting taught in Java/C for an introductory course)
In this case, you need someone that can implement the method as described (hard!), and then you need someone with a setup better than a rented 8xA100 (expensive and not available on many cloud providers) to actually reproduce the model.
About as exciting as Imagen-video, which was released eons ago.