thesz

  > gradient descent isn't good at combinatorial optimisation.

If you convolve your problem with sufficiently wide Gaussian, you can use gradient descent. The approach is called Natural Evolution Strategies [1].

[1] https://en.wikipedia.org/wiki/Natural_evolution_strategy#Nat...

It requires O(N^4) evaluations to compute Fisher Information Matrix for N-dimensional parameterization of the problem in original formulation. But there are closed form solutions and more economical representations of covariance matrix (LoRA, hehe).

Multiply "inference + backwards pass (~2x inference cost) + activations (vram overhead)" by batch size (thousands) to get to the actual RAM and compute cost. Optimizer like ADAM adds only two or three model-sized overhead.

And last, but not least, you need only one hidden layer kept in RAM for inference, but you need all of them (61 for Deepseek models) kept in RAM for computing gradient for one sample.

The difference between training and inference is 1) one have to keep intermediate results for backward pass in training and 2) computation for training double because of the backward pass.

Training is also done over batches, which increase memory requirements by several orders of magnitude. This is why training needs costly compute.

One of the ways out of this unfortunate situation is to use something like Stochastic Average Gradient Descent [1]. Examples there are mostly concerned with regularized logistic regression, which makes problem more or less convex. Neural networks are inherently non-convex. Still, maybe some ideas from there can be utilized in the context of neural networks, like use of estimated Lipshitz constant to derive curvature and appropriate learning step.

  [1] https://www.cs.ubc.ca/~schmidtm/Courses/540-W19/L12.pdf

English is not my native language.

PSP/TSP (Personal and Team Software Processes) and RUP (Rational's Uniform Process, to a lesser extent) are very valuable approaches to software engineering. Please, read about them, they are very interesting.

LLMs are Markov Chains [1]. "Emergent abilities" of LLMs can be explained by decrease of perplexity in text prediction [2].

  [1] https://arxiv.org/abs/2410.02724
  [2] https://arxiv.org/abs/2304.15004

  > central concept or software engineering is architecture patterns.

Both RUP and PSP/TSP do stand on the ground of defect prevention. All sorts of defects, from incorrect sets of requirements to memory corruption.

Architecture patterns can be of help in that regard and they also can be very error-prone, as right now I am in the process of removing a bug introduced through misunderstanding of one rather old singleton.

  > As long as the tests pass and the performance doesn't regress...

AMD had 20+ million tests for their FPUs back in the day of Intel's FDIV bug and ACL2 found bugs in their implementations of floating point computation.

Agentic vibe coding is not an application of ACL2 theorem prover to anything. Agentic vibe coding is an opposite of it, it will make its way to pass the tests with any means necessary, be it patching the code, the tests, or expected results.

   > it's secure

You can't say that before formal verification. Which is an opposite of what vibe coding is.

The steepness of Haskell learning curve is exaggerated.

I have experience where completely unmotivated (due to then ongoing very bad divorce) former Java programmer took a description of CPU for model code generation in Haskell's eDSL and successfully extended it in a month. The model had tricky (at the time) type level programming and type classes with associated types.

I think you may find helpful this [1] almost 20 years old [2] agent.

  [1] https://github.com/augustss/djinn
  [2] http://lambda-the-ultimate.org/node/1178

The context window it requires for AI coding can be as short as half a dozen of tokens.

  > LLMs have a temperature parameter. At zero temperature they are deterministic: they always choose the most likely next token at each step based on what came before and the model weights, and they will always generate the same output given the same input.

https://en.wikipedia.org/wiki/Softmax_function

"A value proportional to the reciprocal of β is sometimes referred to as the temperature: β = 1/kT, where k is typically 1 or the Boltzmann constant and T is the temperature. A higher temperature results in a more uniform output distribution (i.e. with higher entropy; it is "more random"), while a lower temperature results in a sharper output distribution, with one value dominating."

"Temperature" in the context of softmax does not change a "winning" token, it changes how much probable (in the sense of softmax distribution) winning token will be. If the winning token is "New York", it will be a winner with temperature close to 0 and with temperature of 1e9.

The actual selection of the random token is done separately by using inputs outside of the softmax distribution, for example, by using random number generator. I believe most of LLM configs have a seed for the random number generator.

More than that, generation of code in most programming languages is done with the more guardrails such as beam search guided by schema, syntax and semantics.

Have you tried to change your HDL to something more modern like Bluespec System Verilog or, god forbid, anything embedded into Haskell or Scala?

I read that BSV source code is about three times shorter than similar design in Verilog and also has three times smaller defect density (defects per significant line of code). So just by changing the HDL from Verilog to BSV one can have nine (9) times less defects in the design.

Alpha 21064, 1992, was using domino logic [1].

[1] https://en.wikipedia.org/wiki/Domino_logic

There was no synthesis algorithms that would map VHDL or Verilog designs into domino logic elements at the time. I believe that the most work in the synthesis-to-domino-logic area was done at the beginning of current century.

So, DEC's engineers and, I think, Intel's engineers were doing work using schematics well into 21-st century.

  > But it’s not a clear cut that Haskell type system offers more safety guarantees.

You can log operands in your implementation of Rust's Add trait, or compute running sum of one of the operands. In Haskell it is not possible unless you use unsafePerformIO from System.IO.Unsafe.

Haskell's type system controls effects available to the code. This can be used to implement programs adhering to specific formulas of Linear Temporal Logic or implementing a protocol specification, where operations on any given phase and side are restricted by type system.

I used Haskell's type system to prevent crossing of clock domains in the hardware description eDSL. Also, it was of great help in the CPU simulator description, fixing available commands and resources for different CPU models.

Even the logic of Rust's borrow checker was expressible in Haskell as early as February 2009 - there was HList, there was ParameterizedMonad and that's about what one needs for implementation of borrow checker.

They used LTL as non-reduced binary decision diagrams (BDDs, binary trees in essence) to prove that transformers are exponentially more succinct that LTL. Should they allow reductions in LTL expressions, that exponential advantage of transformers would vanish in a puff of smoke, because reduced ordered decision diagrams (ROBDDs) of addition circuits (used to construct their exponential LTL example) are polynomially sized.

How to add reductions to LTL? Allow (parametric) definitions of subformulas. E.g., "let p = ... in xUp/\yUp".

Also, note that they construct transformers, transformers are not trained. Training on any truth table is as hard as one can imagine.

Better food.

Haskell is the best imperative language [1].

[1] https://stackoverflow.com/questions/6622524/why-is-haskell-s...

Haskell have array-oriented programming embedded DSLs, like REPA [2] and Accelerate [3].

[2] https://hackage.haskell.org/package/repa

[3] https://hackage.haskell.org/package/accelerate

The most used (by me) Vector package has boxed (efficient) and unboxed (J-like) arrays. With these arrays having map, fold, scan, fromList/toList, zip and unzip combinators, one can have as terse (in the operator count sense) notation as one wishes for.

  > On the other hand, LLM-generated code comments better than I do, so given a long enough time horizon, it could be more understandable at a later time than code I've written myself (we've all had the experience of forgetting how things work).

Writing and rewriting piece of software performs what is called "spaced repetition" [1].

[1] https://en.wikipedia.org/wiki/Spaced_repetition

You ask questions about code when you implement something and if you cannot answer these questions, you go to code to find answers out and refresh your understanding of it.

For this to work you have to be interested in the understanding of the code and code should be created at the pace you can keep up.

Software engineers usually do create code economically because they need to remember and understand it. Vibe coders do not have this particular constraint, they just do not aim for most understandable code possible. Even if there are more comments in code.

  > LLMs make me significantly faster at writing code I was mediocre or bad at. But when I use it to write code in domains I have more knowledge in I see design and correctness problems all over the place and actively fix them and it slows down my output.

I think a very similar phenomena is called Gell-Mann amnesia effect: https://en.wikipedia.org/wiki/Michael_Crichton#%22Gell-Mann_...

XML: 1996

Perturbation of dataset used for training can introduce adversarial behavior even without adding any other data, and idea is quite simple: you take two batches from the dataset for training and select model with more probable adversarial behavior. The more batches with posterior selection get processed, the more probable adversarial behavior become.

By determining if model gets better or not on a given benchmark, OpenAI selects models against benchmarks, implicitly using them in the training.

Thank you, these posts are very insightful!

> The other, is that models have a blind spot, therefore creating a red team adversary bug hunter agent will not surface a bug if the same model originally wrote the code.

This is very interesting, if true. It follows that one can generate several instances of the code, chose one with the bug and bug will not be found. Mythos can be used to fool Mythos.

BPE that is used in tokenization is very simple: https://en.wikipedia.org/wiki/Byte-pair_encoding

The page does very poor job tokenizing phrase "Noinceolik fiyulnabmed fyvaproldge" into "Noinceolik fiyulnabm ed fyvaproldge", factoring only "ed" suffix. As if made up words such as "noinceolik" are so common they are part of 100K token vocabulary.

The actual application of GPT-5 tokenizer at [1] to my made up phrase results in 14 tokens, only two of them are four characters long and there are tokens containing spaces.

[1] https://gpt-tokenizer.dev/

I will read along, though.

Actually, the euphoric mood disorder may make one hear voices telling to feel great, do good, help all grandmas of the world through the crossing, etc.

The "focus" and "get back to work" parts are hard, though.

To have some confidence in consistency of results (p-value), one has to start from cohort of around 30, if I remember correctly. This is 1.5 orders of magnitude increase of computing power needed to find (absence of) consistent changes of agent's behavior.

If I may (and correctly understand what is going inside Fil-C), it would be not so hard to add support for software transactional memory by adding some library calls.

This will greatly reduce coordination bugs in parallel programs and may even speed things up.

You took a simple path, embedding smaller into larger. What if you need to reduce number of layers and/or width of hidden layers? How will you embed larger into smaller? As for the "addition of same layers" - would the process of "layers to add" selection be considered training?

What if you still have to obtain the best result possible for given coefficient/tokenization budget?

I think that my comment express general case, while yours provide some exceptions.

Thanks, I didn't knew that. I thought that SLOC means "Source Lines Of Code."

That "underappreciated underlying structure or geometry" can be just an artifact of the same tokenization used with different models.

Tokenization breaks up collocations and creates new ones that are not always present in the original text as it was. Most probably, the first byte pair found by simple byte pair encoding algorithm in enwik9 will be two spaces next to each other. Is this a true collocation? BPE thinks so. Humans may disagree.

What does concern me here is that it is very hard to ablate tokenization artifacts.

Change layer size and you have to retrain. Change number of layers and you have to retrain. Change tokenization and you have to retrain.