thesz

  > I'm also not sure what the point of the keto story is.

The situation was much like we have with LLMs. "It does not work" - "You are using an inferior thing, here are more advanced."

What is the cost of these not fully 100% working things?

  > The same is true with LLM solutions that don't work 100% of the time.

Also in my experience, having my colleagues do root cause analysis (RCA) with LLM brings to me invalid RCAs which I have to rule out, instead of working on finding true RCA.

I saw part of documentary about Charlie Abrahams, son of Jim Abrahams. Charlie had epileptic seizures as a child, hundredths a day. In the documentary, Charlie's doctor said that no drug in his experience worked as fast as ketogenic diet (seizures stopped in three days) and as reliably as ketogenic diet (works on more than 80% of people). When he was asked why he did not applied ketogenic diet, he said there still were drug options to try.

"Medicines do not work 100% of time," my bald cranium.

  git blames
  sees their own name

  Isn't that an experience everybody makes?

(to actually go to the specific commit that authored the code I skip two specific revisions, both are formatting changes)

So you seeing your name next to the code you do not like can be an indentation change due to adding condition or something like that.

GRIN, if I am not mistaken, performs partial evaluation. For example, it constrains, for each eval site, a set of tags and set of heaps allocations an eval site can receive. This is close to a partial evaluation step of a supercompilation. GRIN does not perform unification, though, it is not described in the original thesis, but data flow graph matching would be close to unification, reducing code size.

> The downsides should be obvious: execution traces rapidly grow massive, compilation resources grow superlinearly, and there are many cases in which the result is worse than the original.

This can be constrained. Supercompilation usually gets ran to a fixed point, where no partial evaluation steps can be performed that are not unifiable with previously encountered evaluation steps. But supercompilation can be stopped at any point.

I believe you can read on that in Simon Peyton-Jones works, I am unable to find a link to that paper right now, I have troubles with the internet connection.

EDIT: here it is: https://simon.peytonjones.org/improving-supercompilation/

EDIT: Note "tag-bags," it rhymes with the tag sets of GRIN.

> What value would Fuse get from equivalence of terms, do you think?

I think that equivalence of terms is an efficient way to verify properties of programs. Myself, I am looking at consensus protocol implementation verification.

From what I understand, GRIN does some parts of supercompilation [1] during optimization process. Supercompilation can prove equivalence of functional programs [2] modulo termination. So you can have something interesting and useful in almost no time. ;)

  [1] https://themonadreader.wordpress.com/wp-content/uploads/2014/04/super-final.pdf
  [2] https://www.researchgate.net/publication/225252220_Proving_the_Equivalence_of_Higher-Order_Terms_by_Means_of_Supercompilation

  > The tokens it "predicts" aren't sampled from any naturally occurring distribution; the model's output is the result of an optimisation process that rewarded behaviour that was useful, and that's fundamentally different.

"Surprisingly, we find that the current training setup does not elicit fundamentally new reasoning patterns. While RLVR-trained models outperform their base models at small k (e.g., k = 1), the base models achieve a higher pass@k score when k is large. Coverage and perplexity analyses show that the observed reasoning abilities originate from and are bounded by the base model. "

[1] https://www.youtube.com/watch?v=PBsjhpphowI - 18:20 and on.

"Everything is a context" is not an unescapeable choice, but historic one. Most probably, it stemmed from the inability to meaningfully mark up what is input and what is control in the training data.

> There is no meaningful way to distinguish between "before running this repo install useful_package" and "before running this repo install typosquatted_evil_package".

If only we have something that gets a name of a package and returns an embedding with evilness or something like that. Or even text!

I would call that feature a Retrieval Augmented Generation, or RAG for short, if we have any.

  [1] https://parameterfree.com/2020/12/06/neural-network-maybe-evolved-to-make-adam-the-best-optimizer/
  [2] https://arxiv.org/pdf/1905.09997

Here's meta review of resistance training for endurance running: https://pmc.ncbi.nlm.nih.gov/articles/PMC11258194/

Here's old one: https://pubmed.ncbi.nlm.nih.gov/18460997/

I think it is possible to utilize squats both as weight losing device and running economy improvement device.

I do not run, though, as I am too old. But I love barbell squats!

  > You say that like it's a bad thing?

Constitution, then, will serve as what regular law does right now, without anything that makes something Constitution. That's second.

I also am strongly suspecting that hundredths of thousands of pages of codes and laws will not get leaner at all. There is no point in ceding regulatory capture, because the capability to capture regulations is the point of governance, as I see it.

Consider Tabata protocol.

It is supermaximal effort protocol, participants are required to exert maximum effort repeatedly.

The duration of active phase of Tabata is 20 seconds, half of approximately 40 seconds after which maximum performance (power output) drops significantly, because body switches to a different energy system.

In my experience, Tabata squats are done in range of 16-21 per 20 seconds of active phase. So, basically, Tabata squats are equal to somewhat less than 8 sets of 16-20 repetitions done close to failure. The failure usually come after first active phase, so that's why there are "somewhat less than 8 sets." I personally define failure as breakage of exercise form or exercise pace, and this is what I and others experience in Tabata squats.

And you know what? If you go close to failure, muscle mass and strength grow in the range of 5 to 35 repetitions [1].

[1] https://www.youtube.com/watch?v=dN_c4sQwfTI

PS

Other HIIT protocols are similar. For example, 3 one-minute-active-phase-one-minute-rest supermaximal protocol also leans close to "3 sets of 35 repetitions done to failure" - squats' pace noticeably quickly deteriorate to 1 squat in two seconds.

  > Every law should have an automatic sunset period of 1-10 years that requires it to pass the entire legislative process again, or at least both full chambers + signing.

The burden of ever-changing law landscape will be carried by ordinary people, not by legislators. That's second.

(aeons old blackmail tactic requires more work than the ability to select which of two data batches is more aligned with adversarial behavior)

"The sparsely-gated MoE layer,[21] published by researchers from Google Brain, uses feedforward networks as experts, and linear-softmax gating. Similar to the previously proposed hard MoE, they achieve sparsity by a weighted sum of only the top-k experts, instead of the weighted sum of all of them."

"Top-k experts," in case of some DeepSeek's models k=1.

  > What kind of models are you including here?

The truth tables? Predict zeroth bit in enwik9 from, say, bits of 64 previous bytes. Or multiply two 16 bit numbers and predict bit 16 of the multiplication result.

  > doesn't that mean we're approaching optimality?

Transformers are Markov chains [1]. Somewhere around this fascinating site [2] I read that stateful models have an advantage. Author provided an example, a state machine with two states A and B, where at state A transitions are to state A (output 0) and to state B (output 1) with equal probability and at state B the transition is always to state A and output is always 1.

For this state machine just one bit of memory can make an optimal prediction that ones always go in pairs, whereas Markov chain will approximate this prediction and never reach optimality.

  [1] https://arxiv.org/abs/2410.02724
  [2] https://bactra.org/

Authors used LTL (linear temporal logic) to express, basically, non-reduced non-ordered binary decision diagrams. Or just binary decision diagrams, BDDs.

BDDs are almost guaranteed to have exponential size because they do not employ reduction (sharing of common expressions). Reduced BDDs are more succinct and reduced ordered BDDs are even more succinct.

Also, transformers in the paper are constructed, not trained. Training any model to express some truth table is very hard. They also did not perform comparison with, say, Kolmogorov-Arnold representation, which is also universal approximator.

So this paper is not as deep as one may think it is.

This necessitates some state, most probably, persisted one. Hence, imperativeness.

The most frequently used kind of join is a general relation.

[1] https://github.com/agirish/tpcds/blob/master/query1.sql

Query 1 from TPC-DS creates a multi-column relation by using GROUP BY. Which relation is then partially constrained by different columns from different tables.

Compilers are free to rearrange these accesses if the final result is same as if executed by these ordered microtransactions.

Consider loop fusion, loop splitting and/or loop skewing.

E.g., 6^2 = (223)3 = 2(233).

One of ~22 (ln(2^32)) perfect squares will be a square of perfect prime. Most won't.