pizza

FWIW you might be interested in this, generalizing to many types of materialized views over increments of data via an equivalent of the chain rule over streams https://muratbuffalo.blogspot.com/2024/11/dbsp-automatic-inc...

On the potency of language..

We say "good music gives you goosebumps". Very reductively, you could argue the whole point of a musician's entire career is literally just to get good at making your skin hair wiggle. What if we're overlooking the possibility that the presence of a physically-sensed reaction would make us value something more?

I think the future of communication could involve something kind of like that, for better or for worse. Optimizing communications for physical reactions. The value prop may just be too high, if the causal direction of this effect (on salience) is actually real.

I read a news article about Trump's re-election in Nov 2024, where they interviewed one of his supporters - they said something to the effect of "when I heard the news, my whole nervous system just suddenly relaxed." I wouldn't be surprised if forms of communication get more and more optimized, whether via RLHF or recsys likes or some other means, for their ability to target, for lack of a better term, "embodiment goals"

This is also known as "The Fundamental Theorem of Stream Calculus" in stream calculus. Using coinduction for an (infinite) stream sigma, eg

    sigma(0)      = head(sigma)
    sigma'        = tail(sigma)
    (a ++ sigma)' = sigma

https://github.com/KellerJordan/modded-nanogpt is pretty great in that respect

If I'm not mistaken, it may be feasible to build one with the Claude Code sdk

Say what you will but customers pay to not suck it up

Anything with a linter means, at minimum, free verifiable rewards for RL (though whether something parses versus looks good is another story). That, plus, they have more data than anyone, and also it seems somewhat reasonable that stronger models could learn 'more' from a given instance or set of examples.

> 3. There are no applications outside of potentially other pure math research.

I would feel remiss not to say: such statements rarely hold

That's why I'm very excited for the potential metaphysical ramifications of DolphinGemma

This seems like the type of work that, whether or not it itself is profound (which I believe it seems it is), will accelerate the forthbringing of derivative works that will definitely be profound.

ChatGPT, is that you?

Check out tropical algebras

Another one: SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking https://arxiv.org/abs/2306.05426

There was also an hn thread: https://news.ycombinator.com/item?id=36425375

I think this is really about the actively made decision to spend longer with each sensation than you 'normally' would - relative to your own individual baseline. Basically whether this is a tool in the person's metacognitive toolkit that they typically use or not. Like 'the feeling of the touch of a thought' on the mind itself, as opposed to a linear pre-simulation of a performance to play out loud. Or the difference between making what you say out loud a cheap, low entropy, lossy encoding of all that you're thinking, versus gathering all the aspects quietly and needing to find a way linearize all that incidental complexity.

If you want a hard problem that's far from predictable with tangible utility, why don't you work on something that helps humans with human problems, but whee you can still use your talents? Some research that could help understand a disease, or a new kind of medical technology, or something like that. I don't know how well it aligns with your salary goals but it might be meaningful.

As I was reading through this, I realized that there's also something kind of similar but in the 'opposite direction' - thermodynamic computing.

E.g.

- radiation tolerant ML: do ML 'work' steps despite noise

- thermodynamic computing: drive ML work 'steps' 'through' noise

Makes you wonder if there could even be a meta level where you can isolate the noise due to radiation, and pass that as an input to the protection parameters, so you could adjust the robustness in an adaptive manner, or something. Or a diffusion model, where radiation noise is the input noise source!

[0] https://www.normalcomputing.com/post/scaling-thermodynamic-c...

Probably not a top priority but it would be really really cool if this thing had solid t-string support from the jump, to the extent that it’s feasible without actually executing code

I don’t think there is clinical level evidence for it, but there’s this thing called Trauma Releasing Exercise which apparently some people swear by. The idea is that you do extremely taxing, high neuromuscular-recruitment static holds etc - til the point of total muscle control loss that turns into widespread trembling. I would speculate that if a big part of feeling bad is getting stuck in a state where tension becomes a problematic passive default state that is hard to get out of, that a short, hard ‘nudge’ in the form of overwhelming exertion might be enough to ‘reset’. All speculation until proven though.

What’s interesting about the origins of the technique is that the guy thought of the possibility that adults learn to suppress visible trauma reactions - uncontrolled shaking, etc - whereas children and animals wouldn’t, and for whatever reason they would also be able to return to normal more quickly. He wondered if maybe that there was a tangible benefit to the shaking itself, in that it could help perturb out of the traumatized state itself.

Are you sure about that? Try these..

- (1e(1e10) + 1) - 1e(1e10)

- sqrt(sqrt(2)) * sqrt(sqrt(2)) * sqrt(sqrt(2)) * sqrt(sqrt(2))

"if it were a fact, it wouldn't be called intelligence" - donald rumsfeld

Think of it as generating a constantly streaming infinite list of latents. These latents are basically decoded to a tuple [time_until_my_turn(latent_t), audio(latent_t)]. You can train it to minimize the error of its time_until_my_turn predictions from ground truth of training samples, as well as the quality of the audio generated. Basically a change-point prediction model. Ilya Sutskever (among others) worked on something like this long ago, it might have inspired OpenAI's advanced voice models:

> Sequence-to-sequence models with soft attention had significant success in machine translation, speech recognition, and question answering. Though capable and easy to use, they require that the entirety of the input sequence is available at the beginning of inference, an assumption that is not valid for instantaneous translation and speech recognition. To address this problem, we present a new method for solving sequence-to-sequence problems using hard online alignments instead of soft offline alignments. The online alignments model is able to start producing outputs without the need to first process the entire input sequence. A highly accurate online sequence-to-sequence model is useful because it can be used to build an accurate voice-based instantaneous translator. Our model uses hard binary stochastic decisions to select the timesteps at which outputs will be produced. The model is trained to produce these stochastic decisions using a standard policy gradient method. In our experiments, we show that this model achieves encouraging performance on TIMIT and Wall Street Journal (WSJ) speech recognition datasets.

https://arxiv.org/abs/1608.01281

Context:

I've spotted an interesting bug where Claude's citation feature occasionally fails to render properly, exposing this raw markup format:

    :antCitation[]{citations="<hash>"}

When I searched for information about this format online, I was surprised to find instead a bunch of content with the same parsing error left intact. Real dead internet theory stuff, lol.

There are articles on The Guardian for each of those topics. They published this article because it's a quick and easy one to write, and they knew we would talk about it.

Well.. autodiff isn't particularly technically sophisticated. But I doubt that when it was first invented in the 1950s, people could have foreseen that a lot of autodiff put together can write an implementation of autodiff.

If you're interested in more technical approaches, I think they're starting to come together, slowly. I've seen several research directions (operads / cellular sheaves for a theory of multiagent collaboration, the theory of open games for compositional forwards/backwards strategy/planning in open environments) that would fit in quite nicely. And to some extent, the earliest frameworks are going to be naive implementations of these directions.

You raise some good points but re: codecs, I was quite unimpressed with how they handled JPEG-XL.

Looks useful for embedding interpreters

Tensor visualizer app itself already looks pretty interesting

In order of nicheness

- Singular learning theory

- Vector-symbolic architectures

- Homomorphic learning

Hard agree. Whether it's 50 cents or 10 dollars per session, I'm using it to get work done for the sake of quickly completing work that aims to unblock many orders of magnitude more value. But in so far as cheaper correct sessions correlate with sessions where the problem solving was more efficient anyhow, they're fairly solid tips.