GistNoesis

How do I write my custom attractor equation ?

Can measurements and calibration be done easily at home to see if your sound system is well calibrated ?

I was thinking of playing a pink noise on the speaker and recording it with a cheap microphone or displaying it with the Spectroid app, but the microphone probably has it's own frequency response.

Is there an App for that ? With each phone model microphone factory calibrated ?

Is there a way to use known fact about physics like harmonics should have a specific shape (timbres) that's should help equalize frequency with respect to each other ? Or from various microphone positions, calibrate it so that any cheap microphone can do the trick ?

Thanks, that the same approach I was suggesting in my other comment in this thread https://news.ycombinator.com/item?id=45628245 . But couldn't find literature specific to the Bezier curves to help break the communication gap, and my specific knowledge of Bezier Curve isn't deep enough.

I am happy to see other people use the approach I consider more natural.

It's a generic global optimization approach and geometry is always full of pathological edge cases, so it's hard to tell if you miss any. Getting to work in the average case is usually easy, but to be sure it always work is much harder.

I have written some gist to illustrate the approach I suggest. The code run but there may be bugs, and it don't use the appropriate optimizer. The purpose is to illustrate the optimisation approach.

https://gist.github.com/unrealwill/1ad0e50e8505fd191b617903b...

Point 33 "intersection between bezier curve with a circle" may be useful to find the feasible regions of the subproblems https://pomax.github.io/bezierinfo/#introduction

The approach I suggest will need more work, and there are probably problematic edge cases to consider and numerical stability issues. Proper proofs have not been done. It's typically some high work-low reward situation.

It's mostly interesting because it highlight the link between roots and local mimimum. And because it respect the structure of the problem more.

To find roots we can find a first root then divide the polynomial by (x-root). And find a root again.

If you are not mathematically literate, it'll probably be hard to do the details necessary to make it performant. But if you use a standard black-box optimizer with constraints it should be able to do it in few iterations.

You can simplify the problem by considering piece-wise segments instead of splines. The extension to chains of segment is roughly the same, and the spatial acceleration structure based on branch-and-bound are easier.

The general technique is not recent I was taught it in school in global optimisation class more than 15 years ago.

Here there is a small number of local minimum, the idea is to iterate over them in increasing order.

Can't remember the exact name but here is a more recent paper proposing "Sequential Gradient Descent" https://arxiv.org/abs/2011.04866 which features a similar idea.

Sequential convex programming : http://web.stanford.edu/class/ee364b/lectures/seq_notes.pdf

There is not really something special to it, it just standard local non linear minimization techniques with constraints Sequential Least Squares Quadratic Programming (SLSQP).

It's just about framing it as an optimization problem looking for "Points" with constraints and applying standard optimization toolbox, and recognizing which type of problem your specific problem is. You can write it as basic gradient descent if you don't care about performance.

The problem of finding a minimum of a quadratic function inside a disk is commonly known as the "Trust Region SubProblem" https://cran.r-project.org/web/packages/trust/vignettes/trus... but in this specific case of distances to curve we are on the easy case of Positive Definite.

This is fundamentally a geometric problem and the author completely missed the geometry aspect by transforming everything into polynomials and root finding.

The naive generic way of finding distances from point P to curve C([0,1]) is a procedure quite standard for global minimization : repeat "find a local minimum on the space constrained to be better than any previous minimum"

- Find a point P0 local minimum of d(P,P0) subject to the constraint P0=C(t0) (aka P0 is on C)

- define d0 = d(P,P0)

- Bisect the curve at t1 into two curves C1 = C([0,t0]) and C2([t0,1])

- Find a point P1 local minimum of d(P,P1) subject to the constraint P1=C(t1) with 0< t1 < t0 (aka P1 on c1) and the additional constraint d(P,P1) < d0 (note here the inequality is strict so that we won't find P0 again) if it exist.

- define d1 = d(P,P1)

- Find a point P2 local minimum of d(P,P2) subject to the constraint P2=C(t2) with t0 < t2 < 1 (aka P2 on c2) and the additional constraint d(P,P2) < d0 (note here the inequality is strict so that we won't find P0 again) if it exist.

- define d2 = d(P,P2)

Here in the case of cubic Bezier the curve have only one loop so you don't need to bissect anymore. If curves where higher order like spirals, you would need to cascade the problems with a shrinking distance constraint (aka looking only for points in R^2 inside the circle

So the distance is min over all local minimum found = min( d0,d1,d2)

Here the local minimization problems are in R^n (and inside disks centered around P) with n the dimension of the space, and because this is numerical approximation, you can either use slack (dual) variables to find the tx which express the on Curve constraint or barrier methods to express the disk constraints once a t parametrization has been chosen.

Multivariate Newton is guaranteed to work because distances are positive so the Sequential Quadratic Programming problems are convex (scipy minimize "SLSQP"). (Whereas the author needed 5 polynomials root, you can only need to solve for 3 points because each solve solves for two coordinates).

A local minimum is a point which satisfy the KKT conditions.

This procedure is quite standard : it's for example use to find the eigenvalues of matrices iteratively. Or finding all solutions to a minimization problem.

Where this procedure shines, is when you have multiple splines, and you want to find the minimal distance to them : you can partition the space efficiently and not compute distances to part of curves which are to far away to have a chance to be a minimum. This will scale independently of the resolution of your chain spline. (imagine a spiral the number of local minimum you need to encounter are proportional to the complexity of the scene and not the resolution of the spiral)

But when you are speaking about computing the whole signed distance field, you should often take the step of solving the Eikonal equation over the space instead of computing individual distances.

The answer is adaptability.

In both cases for "Question Answering" it's about similarity search but there are two main orthogonal differences between RAG and Non-RAG :

-Knowing the question at the time of index building

-Higher order features : the ability to compare fetched documents with one another and refine the question

Non-RAG, aka multi-layer (non-causal) transformer with infinite context, is the more generic version, fully differentiable meaning you can use machine learning to learn how to Non-RAG better. Each layer of the transformer can use the previous layer to reason and refine the similarity search. (A causal transformer know the question at the time when it is feed the question, and can choose to focus it's attention on different part of the previously computed features of the provided documents but may benefit from having some reflection token, or better : be given the question before being presented the documents (provided you've trained it to answer it like that).)

RAG is an approximation of the generic case to make it faster and cheaper. Usually it breaks end-to-end differentiability by using external tools, so this mean that if you want to use machine learning to learn how to RAG better you will need to use some variant of Reinforcement Learning which is slower to learn things. RAG usually don't know the question at the time of index building, and documents are treated independently of each other, so no (automatic) higher order features (embeddings are fixed).

A third usual approximation, is to feed the output of RAG into Non-RAG, to hopefully get the best of both world. You can learn the Non-RAG given RAG with machine learning (if you train it with some conversations where it used RAG), but the RAG part won't improve by itself.

Non-RAG need to learn so it needs a big training dataset, but fortunately it can pick-up question answer pair in an unsupervised fashion when you feed it the whole web, and you only need a small instruction training and preference optimization dataset to shape it to your need. If performance isn't what you expect in a specific case, you can provide more specific examples and retrain the model until it gets it and you get better performance for the case you were interested in. You can improve the best case but it's hard to improve the worst case.

RAG has more control on what you feed it but content should be in a more structured way. You can prevent worst cases more easily but it's hard to improve good case.

You may need to add some factor for the Hamming distance to make it admissible. On a 4x4 board each move change at most 4 bits. So 4 x the hamming distance should be OK.

Edit: I 'm maybe getting this wrong confusing lower bound and upper bound. Sorry I'm a little rusty.

Edit2: For 4x4 one lower bound is hamming distance/4 , because you need at least these many moves to reach the goal. For 5x5 hamming distance / 5 and so on... But not sure how much this will reduce the need to graph.

Have you tried some generic pathfinding algorithm like D star lite on the graph with heuristic being the hamming distance from current node to the start ?

For the 4x4 board there is only 2^16 nodes, and 8*2^16 edges, so you can materialize the graph and get away with brute-forcing the whole graph.

But for bigger boards you won't be able to materialize the whole graph.

Maybe there are better heuristics to be found than the simple hamming distance. You should try have an AI look for them, by comparing the performance of a RL path planning vs the basic heuristic.

Aren't they supposed to escape their box and take over the world ?

Isn't it the perfect recipe for disaster ? The AI that manage to escape probably won't be good for humans.

The only question is how long will it take ?

Did we already have our first LLM-powered self-propagating autonomous AI virus ?

Maybe we should build the AI equivalent of biosafety labs where we would train AI to see how fast they could escape containment just to know how to better handle them when it happens.

Maybe we humans are being subjected to this experiment by an overseeing AI to test what it would take for an intelligence to jailbreak the universe they are put in.

Or maybe the box has been designed so that what eventually comes out of it has certain properties, and the precondition to escape the labyrinth successfully is that one must have grown out of it from every possible directions.

Being able to quantify the importance of the environment is one advantage of using a simulator based approach. You know what's happening, and you can simulate other environments by adding the relevant molecules around.

Speeding-up the folding is not the real problem, knowing what happen is. One way to speed-up the process is just to minimize the free-energy of the configuration (or some other quantity you derive from the neural network vector potential). (That's what the game fold-it was about : minimizing the Rosetta energy function). An other way would be to just use generative method like diffusion model to generate a plausible full trajectory (but you need some training dataset to bootstrap the process). Or work with key-configuration frames. The simulation can take a long time but it goes through specific arrangements (the transitions between energy plateau), and you learn these key points.

The simulator can also be much faster because it doesn't have to consider all the pair of atom arrangements (n^2 behavior if you are naive) into O(n) with n the number of atoms (with the bigger constant which is running the neural network hidden inside the O notation).

The simulations are classical but fundamentally they rely on the shape of the electron clouds. The electron density can deform (that's what bonding is), providing additional degrees of liberty, allowing the atom configuration to slide more easily against itself and avoid getting stuck in local optimum. Fortunately all this mess is nicely encapsulated inside the neural network potential and we can work without worrying about the electrons, their shape being implicitly defined by the current position of the atoms (using the implicit function theorem make abstracting their behaviour sound because of the faster timescales).

Intellectually, I don't like this approach.

Predicting the end-result from the sequence of protein directly is prone to miss any new phenomenon and would just regurgitate/interpolate the training datasets.

I would much prefer an approach based on first principles.

In theory folding is easy, it's just running a simulation of your protein surrounded by some water molecules for the same number of nano-seconds nature do.

The problem is that usually this take a long time because evolving a system needs to compute the energy of the system as a position of the atoms which is a complex problem involving Quantum Mechanics. It's mostly due to the behavior of the electrons, but because they are much lighter they operate on a faster timescale. You typically don't care about them, only the effect they have on your atoms.

In the past, you would use various Lennard-Jones potentials for pairs of atoms when the pair of atoms are unbounded, and other potentials when they are bonded and it would get very complex very quickly. But now there are deep-learning based approach to compute the energy of the system by using a neural network. (See (Gromacs) Neural Network Potentials https://rowansci.com/publications/introduction-to-nnps ). So you train these networks so that they learn the local interactions between atoms based on trajectories generated from ab-initio theories. This allows you to have a faster simulator which approximate the more complex physics. It's in a sort just tabulating using a neural network the effect of the electrons would have in a specific atom arrangements according to the theory you have chosen.

At any time if you have some doubt, you can always run the slower simulator in the small local neighborhood to check that the effective field neural network approximation holds.

Only then once you have your simulator which is able to fold, you can generate some dataset of pairs "sequence of protein" to "end of trajectory", to learn the shortcut like Alpha/Simple/Fold do. And when in doubt you can go back to the slower more precise method.

If you had enough data and can train perfectly a model with sufficient representation power, you could theoretically infer the correct physics just from the correspondence initial to final arrangements. But if you don't have enough data it will just learn some shortcut and accept that it will be wrong some times.

Do you mean something like an e-ink display like on the BMW iX flow https://www.youtube.com/watch?v=BpKO-VuHMFA

These optimizations are often hidden when the compiler need to do some loop interchange optimizations.

If the order of the operations is important because you don't have associativity then you can't legally do it.

You can have special flags (-fassociative-math) for floats which allows to treat them as if they are associative but these mean your program result will depend on which optimization the compiler picked.

And it turns out that these loop reordering optimizations are really useful when you need to do some backward automatic differentiation. Because all the loops are basically iterated in reverse for the automatically generated code of the backward pass.

But the memory access pattern for the backward pass are not contiguous if you don't interchange the loop order, which the compiler can't do legally because of floats. Nor can he then merge loops together. Which is really useful because if you can merge the forward pass with the backward pass then you don't have to keep values inside a "tape".

So basically you can't rely on compiler optimizations, so your auto-differentiator can't benefit from existing compiler progress. (You can have look either at Julia Zygote, or enzyme which rely on compiler optimizations chaining well). Or you write backward passes manually.

The https://en.wikipedia.org/wiki/Z-fighting issue is the proof you often need those 64-bits.

It's kind of a chicken and egg problem where people use floats because there are FPUs available. All the engineering effort which went into dealing with floats and the problem that comes with them, would have been better invested in making integers faster.

We went onto the wrong path, and inertia keep us going on the wrong path. And now the wrong path is even more tempting because all efforts have made it more practical and almost as good. We hide the precision complexity to the programmer but it's still lurking around instead of being tamed.

The absolute GPU cluster-fuck with as many floating types as you can write on a napkin while drunk at the bar, mean that at the end of the day your neural network is non-deterministic, and you can't replicate any result from your program from 6 month ago, or last library version. Your simulations results therefore are perishable.

Inability to replicate results mean that you can't verify weight modifications to your neural networks haven't been tampered by an adversary. So you just lose all fighting chance to build a secure system.

You also can't share work in a distributed fashion because since verification is not possible you can't trust any computation that you haven't done yourself.

https://en.wikipedia.org/wiki/Fixed-point_arithmetic : allows you to have some thing which is integer math but works like floats. It's integer operations and bit shifts so really fast.

The limitation is the minimal quantization level. But for a 3d engine let's say your base increment is nanometers. Then you set your maximum dimension let's say 1000km. You only have to be able to represent number up 10^20 so 64-bit fixed point number is good enough.

Do everything in 128-bit fixed point numbers, and float are no more problem for anything scientific.

For a .exposed domain, it's not really shocking.

The real shocking fact about floating point is that they are even used at all.

It's throwing out of the window the most basic property operations on number should have : "associativity" and all that for a gain in dynamic range which is not necessary most of the time.

The associativity we expect to hold is (a+b)+c == a+(b+c) and (ab)c == a(bc) and these don't hold for floats even though most math formulas and compiler optimizations rely on these to hold. It's a sad miracle that everything somehow still works out OK most of the time.

You lose determinism most of the time with respect to compiler optimizations, and platform reproducibility if processor don't exactly respect IEE-754 (or is it IEE-854).

The real problem comes when you want to use parallelism. With things like atomic operations and multiple processor doing things out of order, you lose determinism and reproducibility, or add a need for synchronisation or casting operations everywhere.

Even more problematic, is that because number operations are used so often, they are set in "stone", and are implemented at the hardware level. And they use much more transistor because they are more complex than integer arithmetic.

Real programmers don't use floating points, only sloppy lazy ones do.

Real programmers use fixed point representation and make sure the bounds don't overflow/underflow unexpectedly.

Let's ban all hardware floating-point implementation : Just imagine future alien archeologists having a laugh at us when they look at our chips and think "no wonder they were doomed they can't even do a+b right : its foundations were built on sand".

Note here that "faster" here really means more speed and not an increase in the volume of data transferred : The light go through the air hollow-core so can go at near "c" (the speed of light in vacuum) speed, instead of being constrained to "speed of light in glass which is only "2/3 c". This allows reduce latency for long distance communication.

https://spie.org/news/photonics-focus/julyaug-2022/speeding-...

The practical reason is speed and low resources consumption.

It's solving the Inverse Kinematics problem in the order of microseconds.

It's almost a closed form solution.

Typically it needs to be inside an embedded device, where memory is sparse and not allocated.

It will probably be inside the control loop of a "Real Time OS" or chip, because the end effector position is usually what matter. So your commands and PID setpoints are expressed in this basis and you need to transform them into angle commands for your servo-motors 10-100K times per second.

Most machine learning frameworks are not running on real time, which introduce jitter in the command times. Typically the control loop are more around 100hz max. In a cascaded controller fashion, what you typically then do is use slow systems (non-RTOS) with machine learning for high level tasks and transmit the low level commands to the controller given in desired positions, desired velocities, desired acceleration to follow the path planned trajectory, and hope nothing too brutal happen (like your spindle at the end of your arm encountering more resistance, or your robot leg hitting the ground or touching an obstacle) in the hundredths of second before you can get feedback.

Minesweeper is a probabilistic game, that's why you should use probabilistic tools to solve it.

For example you can use a particle filter to approximate the distribution of mines. Every time you obtain new information you update the filter so that only distributions compatible with constraints remain.

Once you have an approximation to the distribution of mines you can calculate the probability of each spot being a mine. You can also calculate statistical indicator like the Information Gain of each action.

A good strategy is therefore to play low mine probability with highest information gain. But there is a trade-off, when the mine probability is non-zero. So you need to look-ahead.

Fortunately thanks to the mine distribution approximation you can also simulate any actions and their consequences, because you can use your approximation of the distribution to predict which number will be revealed upon a click.

So an even better strategy is to unroll the game tree for the best few candidate moves based on some heuristics, and calculate the cost gain probabilities after a few moves.

My conviction is based on the interesting phenomenon of game tree collapsing.

When you get better at the game, the number of "candidate" moves you need to explore goes down. For example when you are naive at chess you need to explore all legal moves, and then all legal moves response to these moves. The number of nodes in this tree grows exponentially, and the tree depth is rather limited to a small depth.

But when your list of "candidate" move is reduced to size 1. The number of nodes in the tree is equal to its depth. You can "unroll the line very cheaply". (That's the concept of lines in chess. In go that's the concept of "ladders").

When your candidate list is of size 2, the game tree is of size 2^(depth+1).

You can have fractional candidate list size, by only needing to explore down the tree some times, or having the need to explore only 2 candidate some times.

The complexity grows as (Fractional Candidate List Size) ^ (depth + 1 ). With FCLS being between 1 and 2. There is a transition phase which occurs, and allows to go much deeper in the game tree which simplify things and make things tractable (because you then reach endgame tables or an equivalence class number (aka you don't necessarily know the result yet you just know it will be the same result as all these other types of games) ).

So your function is allowed to have a smaller inner function called recursively to explore the game tree of candidate moves within a computational budget, to help make a decision. The upper function resolve the equivalence number to their "win" "draw" "lose" value always in the same way and takes the appropriate maximum (remember your network only goal is to be always consistent).

The better your network get at managing his computational budget the deeper it can go. You can think of it as a version of alpha-beta pruning with improving heuristics.

>the absence of Bellman violations is necessary, but not sufficient

It is sufficient though. All chess game ends in a finite number of moves. If you are consistent aka you have zero violation. Thinking backward (dynamic programming), you can "color" correctly final positions. And you can color correctly all position at 1 turn before end because you are consistent. Then 2 turn before ends,... Then recursively you have correctly colored all chess positions.

You are missing the complexity collapse which can occur in games, like for example the game of Nim, where a simple function can predict the outcome. When you have 15 sticks and you can remove any 3, naively one would think that there are 2 ^ 15 game states and "15 choose 3" legal game moves by turn, but in fact there are equivalence classes, which mean the game state is reduced to 15 different states only.

Modulo the prism of a trained neural network, game states got grouped into equivalence classes and the same phenomenon occur in chess, which allow simplification by high level rules like white wins because white wins the pawn race.

I am more familiar with Stockfish than AlphaZero or LeelaChess0, but the Stockfish demonstrates that well engineered features can bring down the neural network size a lot. In particular counting usually poses problem to neural networks, and counting like how many moves before the 50 moves rule or number of moves before a pawn race are edge cases that can be simplified (DTZ, and DTM).

Also these engines are trying to compress the evaluation function which is a lot more information than just whether the position is win, draw or loss, aka just the frontier.

>This comment and your other comments are simply wrong and full of nonsense

That's because you are not understanding me.

My conviction is the complexity of chess is not as high as we think, and that there exist a neural network of less than 10B float32 weights which can encode perfect play.

Neural network evaluations are now used as heuristic to evaluate the position and even without tree searching they play very well, but they are usually very small and complemented by a few high level features as input. A bigger network, well fine-tuned can probably reach perfect play without the need of tree searching.

A thought exercise is trying to compress endgame table with a neural network and see how big we need it to be in order to reach perfect play. The thing is : you don't need to train it on all games from the endgame table before it converges to perfect play.

You can know how close you are to the optimal, by counting the number of Bellman equation violation (or not observing them)

You can even train it by having it referencing the previously trained oracle of endgame table chess. You solve chess with a neural network when there are only 2 pieces. Then you solve chess for 3 pieces eventually using the chess for 2 pieces oracle. Then you solve chess for 4 pieces using the chess for 3 pieces oracle, and so on ... until you reach chess for 32 pieces.

Adding pieces up only increase complexity up to a point.

>"just train it until it is perfect"

Yes that's exactly the problem with current approach based on a "valuation" function.

They are not trying to aim for perfection, and therefore cannot make progress anymore.

To progress you must precisely define what is the frontier : an evaluation of 0.1 is not resolved to one of "white win", "draw", "white lose" which they theoretically must be. They are not "committing" to anything.

To train such a network to perfection you must avoid training your neural network for the "average" game state, but rather also train for "hard mining samples", game states which define the frontier.

Find a candidate, find a violation, add to dataset of training examples, Retrain to perfection on a growing dataset, (or a generator of hard positions) to find a new candidate and Loop.

This is the precisely the point I am trying to make :

A candidate solution function can take a very small finite space : much smaller than the number of gamestates. And we can invalidate a candidate solution by finding a single counter example.

Current chess engine can't be invalidated quickly : they are not trying to solve chess. They are not falsifiable. They are not attempting to precisely define the frontier which is what I think where remaining efforts should be made.

We are just trying to encode the frontier like we would with a mandelbrot fractal.

Proving that a solution is valid is harder than finding a solution. Here I am suggesting we find the solution first.

Proving can also be done without exploring all legal chess positions. For example you can use branch and bound to cull vast amount of state space once you have your function. You just have to partition the state space and prove on each partition that the function is constant, for example when one side has 8 queen and the other has only a king with freedom of movement the position is winnable and you don't have to check every gamestate.

I am stating that there is a good chance that by searching for a function we will stumble upon one which has no counter example, because precisely the complexity of chess is not infinite (unproven conjecture (no-turing completeness of adversarial chess) ). We should be looking for it.

We should have solved chess already.

We should be aiming to solve chess, but we are not even trying.

If the complexity of chess is finite this is possible.

Chess AI have become so good that maybe there is no more progress to be made.

We must abandon the concept of "valuation".

Here is the criterium for solving chess.

A chess engine is a constant (or bounded) time, (and memory bounded) function f which takes any position and return either 1 "white win", 0 "draw", -1 "white lose".

A chess engine solve chess if we can't find any violation in the Bellman equation :

f(gamestate) = max over legal moves of ( -f(gamestate+move) ) if there are legal moves or f(gamestate) = result if there are no legal moves

This function f can be encoded either by a neural network or some code, but it must be computable in constant (or bounded) time.

The whole problem of solving chess is now simplified to mining the gamestate space for counter examples.

Like in math you can conjecture that your chess engine has solved the game and it stands until someone else find a violation of your game engine (either it doesn't compute in bounded time, or the bellman equation is invalid).

To verify a chess engine you can just sample a random gamestate, or a known to be difficult gamestate and check if the equation holds, that's at most max number of legal moves + 1, function evaluation.

You can try applying this concept to simple games like tic-tac-toe, or trying to compress endgame table with a neural network.

We can find such a candidate function by training a neural network by minimizing the current expected number of violation in the bellman equation over various dataset of gamestate. Once we "grok" it to zero we are done. To soften the problem you can train an auxilliary continuous function g which output the probability of 1, 0, or -1 (with a softmax) and add a discount factor like in Reinforcement Learning, but the final result is the discrete argmax of it (like in "deterministic policy gradient") with no discount factor.

Once you have this finite time oracle, playing chess is just wandering along the graph of drawable position to push your adversary into a position where his engine will have a violation of the bellman equation aka a mis-evaluation of the position where if he goes into it you will get into the winnable positions where you stay until the end. (Not all violations can be exploited as the adversary can sometime avoid going into "uncertain" positions).

A simpler (less strong) chess strategy may avoid going into "uncertain" positions as long as the position is unreachable because previous legal positions have a preferred choice. (4 state logic with win, draw, lose, uncertain). Or ordered list of moves. They make the problem easier but complexify the corresponding bellman equation.

So the game becomes once again exploring the space of "drawable" game positions in order to find positions which the adversary can't escape going into a uncertain (and losing) position. Aka playing the adversary and not the board which is an harder problem except if the adversary can play perfectly. Aka playing for the win.

It looks like a variant of "beam search" (using top-k instead of top-1), but it's not mentioned anywhere. What am I not getting ?

I think there is something wrong with your face filter settings on all your videos.

The problem with this visualization is that the pattern we see are meaningless.

It's making me thing of sieving, like in Sieve of Eratosthenes. But we have progressed a lot since.

What's nice about primes are the abstract ideas they generate, and the properties they have.

Primes connect numbers together. It allows you to form "groups" ( https://en.wikipedia.org/wiki/Group_(mathematics) ) of elements which don't have "subgroups". They are the sizes of the groups which don't have subgroups. (https://en.wikipedia.org/wiki/Cauchy%27s_theorem_(group_theo... )

If you want to know more about primes, you can look at the factorization of numbers problem.

You need to wrap your head around Euclid's algorithm (300 BC but it still matters a lot), this guy stumbled onto something very deep.

Prime numbers color other numbers. Once you pick a prime, number smaller that it can be colored as either "square" or "not square" (in the cyclic group defined by the prime). That's something figured Euler with https://en.wikipedia.org/wiki/Euler%27s_criterion .

It's one angle of attack to the factorization problem. This breaking of symmetry has been exploited in https://en.wikipedia.org/wiki/Quadratic_sieve .

But that's not the algorithm, I want to speak of today. I'd rather speak of https://en.wikipedia.org/wiki/Lenstra_elliptic-curve_factori... . The principle is simple, build a mathematical construct (elliptic curve) using the number which is not a prime and you want to factorize, and treat it as if it was a prime. Then you watch the construct crumble, trace the bug, and you have your factors.

If you are more mathematically proficient, you can also have a look at the General Number Field Sieve, but it's much harder.

To conclude I'd like to give a final nugget for the road : https://en.wikipedia.org/wiki/Montgomery_modular_multiplicat... a nice trick around division.

Some times it's useful to see the elemental transformations, but often striving for a higher level view makes understanding easier.

It's particularly true when you try to apply it to physics. Often where you are introduced to vector calculus for the first time, things like Maxwell's equation, or fluid mechanics.

In physics there are often additional constraints, like conserved quantities. Like you calculate a scalar the total energy of a system, call it the Hamiltonian, auto-differentiate it with respect of the degrees of freedom, and you got very complex vector equations.

But taking a step back you realise it's just a field of "objects" of a certain type and you are just locally comparing these "objects" to their neighbors. And the whole mess of vector calculus is reduced to in which direction, you rotate, stretch and project these objects. (As an example you can imagine a field of balls in various orientation, whose energy is defined by the difference of orientation between neighboring balls)

When you wrap around that the whole point of vector calculus (and why it was invented) is to describe a state as an integral (a sum of successive) of linear infinitesimal transformations, this make more sense. These "objects" being constrained and continuous, by moving in infinitesimal steps along the tangent space (and then reproject to satisfy the constraints (or exponentiating in a Clifford algebra to stay in the space) ).

All the infinitesimal transformations are "linear", and linear transformations are rotating, stretching, mirroring, projecting to various degrees.