slickytail

You seem to misunderstand what full screen mode is. It doesn't just make the terminal fullscreen, it causes pi to maintain its own scrollback buffer and a UI around it, rather than letting the terminal emulator handle scrolling (ie, letting the session context actually live in the terminal history)

Structured decoding limits next-token probabilities to ensure valid JSON. The main issue is that if the model puts a substantial probability on an invalid token, then it was already confused, and in that case, you don't actually want whatever the next-most-likely valid token is: even if it's syntactically valid, it's likely semantically erroneous.

Reminder that job postings for jobs in California are legally required to include a salary range.

You forgot the best line: "I don't know how the US economy works, much less some kind of self-sustaining one".

The idea of Simulacrum comes from Baudrillard. His essay "Simulation and Simulacra" is highly recommended for understanding what is so strange about the modern economy.

I think it would, for all practical purposes, be impossible to determine an optimal warrior, even at very small core sizes. Not only is the search space huge but the evaluation function can take unbounded time to resolve. We should consider the halting problem embedded inside the optimization target as a clue to the problem's difficulty.

As a non Emacs user, I would be really interested in a full writeup of how this works.

The site has a little game where you click to dismiss as many cookie-consent popups as possible, in 30 seconds. I suppose if you can't see the cookie consent popups, then at the end of the timer you just have zero points.

In the words of Kronecker: "God created the integers, all else is the work of man."

Two different groups of people commenting different things.

In what sense is this 64KB? Clearly there's more than 64KB of code in the repo. And since it's typescript it's not like there's a binary that could be 64KB.

In a discrete Fourier transform, the number of frequencies you get out is the number of datapoints you have as input. This is because any frequencies higher than that are impossible to know (ie, they are above the sampling frequency).

But in the continuous Fourier transform, the output you get is a continuous function that's defined on the entire real line.

Traditional suicide is incredibly stigmatized; ending one's life manually is a huge trauma to place on loved ones. The benefit of MAID is that it's dignified, and won't leave families searching for answers after a death.

The transformer was a major breakthrough in NLP, and it was clear at the time of publishing that it would have a major impact. But I will add that it is common in the Deep Learning field to give papers catchy titles (see, off the top of my head: all the YOLO papers, ViT, DiT, textual inversion). The transformer paper is one in a long line of seminal papers with funny names.

I think the biggest sign of a potential crash is that Meta has frozen AI hiring. Such a quick reversal from just a few months ago.

I stand corrected! Thanks.

Source for this? My understanding was that this was true for dalle3, but that the autoregressive image generation just takes in the entire chat context — no hidden prompt.

The memory bandwidth on an H100 is 3TB/s, for reference. This number is the limiting factor in the size of modern LLMs. 100GB/s isn't even in the realm of viability.

All of this only really applies to LLMs though. LLMs are memory bound (due to higher param counts, KV caching, and causal attention) whereas diffusion models are compute bound (because of full self attention that can't be cached). So even if the memory bandwidth of an M3 ultra is close to an Nvidia card, the generation will be much faster on a dedicated GPU.

The relative unimportance of the exact SDPA attention in use in modern transformers is already known: https://arxiv.org/abs/2111.11418

The FFN, normalization, and residual connections are absolutely irreplaceable -- but attention can be replaced with almost any other layer that shares information between tokens, such as pooling, convolution, random mixing, etc.

The linked wikipedia article below says that they asked you for your email password specifically -- is there any evidence that they would try to use your linkedin password itself?

Generally the problem with this type of argument is that the two sources are not volume-matched. Try out an ABX test here, of lossless vs various lossy codecs: https://abx.digitalfeed.net/