DevelopingElk

They do work, but transformers work better on the accuracy/compute tradeoff curve. Transformers let far away positions in the sequence exchange info in 1 step, while a dilated CNN will need many layers. Additionally, Transformers can be fairly efficiently trained.

I do think if Transformers weren't invented you would still be able to train powerful language models, but they would take more ram and be slower so nobody does this.

CPS is a way of embedding imperative computation into an FP language. I think they built a mini compiler for their binary relation language, which is then Jitted by Julia.

Russian one time pads were frequently broken. How? They were inconvenient to distribute, so people reused them.

Symmetric cryptography is safer because one key can easily protect as much data as you need.

One of the data center's cooling loops broke.

I ran into this problem where I wanted to generate balanced trees for test cases, and I decided to crack it with math.

Let X = the number of remaining open parenthesis required to reach your target length, and Y = the number of remaining close parenthesis.

Let P(X) = X/(X+Y) * (1 + 1/(Y-X+1)). Generate an open parenthesis with probability P(X), and otherwise a close parenthesis.

This will efficiently generate an unbiased random plane tree. The reason this works has to do with counting the valid ways to complete the plane tree, using a calculation similar to the one in the article.

His claim is that we exp-minus-log cannot compute the root of an arbitrary quintic. If you consider the root of an arbitrary quintic "elementary" the exp-minus-log can't represent all elementary functions.

I think it really comes down to what set of functions you are calling "elementary".

According to the article “This work effectively rules out explanations of the Hubble tension that rely on a single overlooked error in local distance measurements". So any systemic errors would need to affect multiple measurement types.

Good? Bad? Doesn't matter as long as you had fun.

Have you tested this GCs performance? Sometimes a baby GC can be fast enough.

The issue with Regex for parsing is it can't handle balanced parentheses. https://en.wikipedia.org/wiki/Regular_expression. More generally, they can't handle nested structure. Context free grammars are the most natural extension that can. It adds a substitution operator to Regex that makes it powerful enough to recognize nested structure. So, Regex would be reinvented if history was rerun, but so would Context Free Grammars. Part of the complexity in parsing is attaching semantic meaning to the parse. Regex mostly avoids this by not caring how a string matches, just if it matches or not.

Now, I do agree that LR grammars are messy. Nowadays, they have mostly fallen from favor. Instead, people use simpler parsers that work for the restricted grammars actual programming languages have.

IIRC there is some research into formalizing the type of unambiguous grammar that always uses () or [] as nesting elements, but can use Regex for lexing.

I'm fairly sure this was human written.

Rust's discussion boards has an idea of "keyword generics" for expressing some of these concepts. The idea is that a function can be generic over const, async or some other keyworded effect. I like this description. It shows the benefits without too much theory.

Yours might go a little less into the details, but its really clear and I like the diagrams and explanation around glitch hazards. Please do follow up on your tangents if you have time.

I have commented once or twice on articles being AI generated. I don't put them when I think the writer used AI to clean up some text. I added them when there are paragraphs of meaningless or incorrect content.

Formats, name collisions or back-button breakage are tangential to the content of the article. Being AI generated isn't. And it does add to the overall HN conversation by making it easier to focus on meaningful content and not AI generated text.

Basically, if the writer didn't do a good job checking and understanding the content we shouldn't bother to either.

This is written by an LLM account. My guess is this article was created with some human guidance too, but the profile shows LLM patterns.

The start of the article is good, but it starts to sound like LLM staring at the "Why this maps to Genetic Algorithms?" section. Is that the case?

By the definition of a cryptographically secure PRNG, no. They, with overwhelming probability, produce results indistinguishable from truly random numbers no matter what procedure you use to tell them apart.

I worked on a team deploying a service to European Sovereign Cloud (ESC). Disclaimer - I am a low level SDE and all opinions are my own.

AWS has set up proper boundaries between ESC and global AWS. Since I'm based out of the US I can't see anything going on in ECS even in the service we develop. To fix an issue there we have to play telephone with an engineer in ESC where they give us a summary of the issue or debug it on their own. All data is really 100% staying within ESC.

My guess is that ESC will be less reliable than other regions, at least for about a year. The isolation really slows down debugging issues. Problems that would be fixed in a day or two can take a month. The engineers in ESC don't have the same level of knowledge about systems as the teams owning them. The teething issues will eventually resolve, but new features will be delayed within the region.

The consequence of Noether's theorem is that if a system is time symmetric then energy is conserved. On a global perspective, the universe isn't time symmetric. It has a beginning and an expansion through time. This isn't reversible so energy isn't conserved.

RL before LLMs can very much learn new behaviors. Take a look at AlphaGo for that. It can also learn to drive in simulated environments. RL in LLMs is not learning the same way, so it can't create it's own behaviors.

Oof, I think that you are right. The issue with Futurelock is a failure of liveness, where the Future holding the lock doesn't get polled. tokio::join! would keep it alive and therefore my suggestion would mistakenly panic.

Yeah, the true fix is probably some form of the fabled Linear types/Structured concurrency where you can guarantee liveness properties.

Disclaimer - I'm not a Tokio dev so what I say may be very wrong. Some definitions:

    Future = a structure with a method poll(self: Pin<&mut Self>, ...) -> Poll<Self::Output>; Futures are often composed of other futures and need to poll them. 


    Tokio task = A top-level future that is driven by the Tokio runtime. These are the only futures that will be run even if not polled. 

My understanding is that Tokio async locks have a queue of tasks waiting on lock. When a lock is unlocked, the runtime polls the task at the front of the queue. Futurelock happens when the task locks the lock, then attempts to lock it a second time. This can happen when a sub-future of the top level task already has the lock, then it polls a different future which tries to take the lock.

This situation should be detectable because Tokio tracks which task is holding an async lock. One improvement could be to panic when this deadlock is spotted. This would at least make the issue easier to debug.

But yes, I think you are right in that the async mutex would need to take the future by value if it has the capability of polling it.

It is well known that Rust's async model can lead to deadlocks. However, in the futurelock case I have come around to blaming the Async Locks. The issues is that they are not "fair" in that they don't poll the future holding the lock. There may be some other tradeoffs that would happen if the locks were in some way "fairer" but I think they should be explored.

I would say, though, that for most programs any one of the most popular languages would do the trick. By this I mean Java, Go, C#. Javascript, Python, C++. All of those are general purpose multi-paradigm languages that you can code almost anything in.

That being said, some programs can only be written in one of those. Browser code is JS exclusive, low-level needs C++, secure code needs not C++. Machine Learning needs Python and high performance can't use Python. Some Windows things need C#. Those cases are the obvious ones where there is basically no choice. Beyond those, it is mostly about the team.

My view of this is that its closer to the basic 2 lock Deadlock.

Thread 1 acquires A. Thread 2 acquires B. Thread 1 tries to acquire B. Thread 2 tries to acquire A.

In this case, the role "A" is being played by the front of the Mutex's lock queue. Role "B" is being played by the Tokio's actively executed task.

Based on this understanding, I agree that the surprising behavior is due to Tokio's Mutex/Lock Queue implementation. If this was an OS Mutex, and a thread waiting for the Mutex can't wake for some reason, the OS can wake a different thread waiting for that Mutex. I think the difficulty in this approach has to do with how Rust's async is implemented. My guess is the algorithm for releasing a lock goes something like this:

1. Pop the head of the wait queue. 2. Poll the top level tokio::spawn'ed task of the Future that is holding the Mutex.

What you want is something like this

For each Future in the wait queue (Front to Back): Poll the Future. If Success - Break ???Something if everything fails???

The reason this doesn't work has to do with how futures compose. Futures compile to states within a state machine. What happens when a future polled within the wait queue completes? How is control flow handed back to the caller?

I guess you might be able to have some fallback that polls the futures independently then polls the top level future to try and get things unstuck. But this could cause confusing behavior where futures are being polled even though no code path within your code is await'ing them. Maybe this is better though?

DynamoDB is working on going cellular which should help. Some parts are already cellular, and others like DNS are in progress. https://docs.aws.amazon.com/wellarchitected/latest/reducing-...

The start is good, but later paragraphs appear to be mostly LLM.

Video processing is one of those things that need caution when doing serverlessly. This solution makes sense, especially because S3s durability guarantees aren't needed.

TLDR - this RNG is completely and totally broken.

First, I don't think the error term is contributing much to the solution. It almost never affects the high bit. In addition, it isn't fed back into updating the secret vectors, so I think an analysis can pretend it doesn't exist.

The nonlinear step where each value is squared looks questionable to me. You will only produce quadratic residues (https://en.wikipedia.org/wiki/Quadratic_residue) when you square the numbers. This roughly halves the number of possibilities.

So what this really boils down to is this:

You have a matrix G and a vector s and a prime p. You repeatedly compute s' = Gs (Hadamard) Gs mod p. Each time you run this step you are projecting into a space with dimensionality (p/2)^N from a space p^N. My guess is that most operations will get trapped into short cycles.

Using you example values, after 10 iterations it gets to [9, 16, 13, 8]. This then repeats with a cycle length of 20. Given 4 values with p = 17 you could get up to 83520 values before repeating.

In some random tests, 6 values with p=97 enters a cycle at iteration 3802 even though there were 832,972,004,929 values.

6 values with p=271 enters a cycle at iteration 166,684 even though there were 396,109,944,105,121 values.

AWS, for the ultimate backup, relies on a phone call bridge on the public phone network.

For DynamoDB, I'm not sure but I think its covered. https://aws.amazon.com/dynamodb/sla/. "An "Error" is any Request that returns a 500 or 503 error code, as described in DynamoDB". There were tons of 5XX errors. In addition, this calculation uses percentage of successful requests, so even partial degradation counts against the SLA.

From reading the EC2 SLA I don't think this is covered. https://aws.amazon.com/compute/sla/

The reason is the SLA says "For the Instance-Level SLA, your Single EC2 Instance has no external connectivity.". Instances that were already created kept working, so this isn't covered. The SLA doesn't cover creation of new instances.