winwang

Using it to run easily-scalable Spark clusters. Previously used it for large distributed builds. It's been pretty great (even if annoying at times).

Alright, so we're spherical cow-philosophers... (jk, I like your point!)

4090 tensor performance (FP8): 660 teraflops, 1320 "with sparsity" (i.e. max theoretical with zeroes in the right places).

https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvid...

But at these levels of compute, the memory/interconnect bandwidth becomes the bottleneck.

There was a script I once saw for Twitter where it blocked all users within some follower/followee radius of a set group of people.

For public discourse, I wonder what the right level of agreeable "echo-chambering" is.

The power of having an API that allows usage of the Free monad. And in less-funny-FP-speak, the power of allowing the user write a program (expressions), that the sufficiently-smart backend later compiles/interprets.

Awesome! Didn't expect such a vast difference in usability at first.

I was following you up until that first comma.

The controller being decoupled is an extremely interesting idea! Makes a lot of sense. I wonder if it includes a (nontrivial) cache.

Until I get around to it, there's this blog post I wrote: https://wiwa.substack.com/p/can-we-10x-rust-hashmap-throughp...

Not an "expert" on GPU hash tables, but I'm mildly aware of other hashmap things like a novel(?) hash-indexed sorted array (HISA): https://arxiv.org/html/2311.02206v3

Makes me consider writing a post on misconceptions of GPU computing, such as requiring the problem to be fully data-parallel.

Two programs which are semantically equivalent are not simply the same. See: bubblesort vs mergesort. (Yes I'm relying on curry-howard isomorphism here).

Didn't know Steve Blank was looking at quantum computing! That's cool.

I had a brief run-in with error-correcting gates. My research topic was about a simple bit-flip operation (quantum dot qubits) under random "telegraph" noise -- different laser pulse shapes result in different error rates which are actually a function of time (on the scale of the pulse itself). Point being that even given a particular physical qubit framework, individual gates are actually a family of (physical) operations with different error rates (and other features, of course) to be optimized for the same logical operation. Not too unlike the idea of NAND vs NOR flash.

Personally, I feel that's slightly "good" in the sense that there are many paths we can take to get quantum computation "good enough".

Hmm, I should say "the numerical value of pi in base 10" (or really any rational base), even if we were to weaken that with the qualifier "a to arbitrary degree of precision". We know pi in the sense of "a unique real number satisfying many useful properties".

Considering that we don't know the value of pi (not that we could write it out nor read it), I'm not sure your definition of "measure" is the same as mine or most people's.

Yeah, that makes sense now. I think, personally, I've simply seen that design around a bunch, but great on you to question it and call it out -- also plausible that my own headcanon doesn't check out.

I like your take, and I propose there is another kind of division: function "coloring". An async "effect" type (e.g. futures, or even just 'async function') signals to programmers that there is some concurrency stuff going on around the place(s) you use it and that we need to talk about how we're going to handle it. i.e. rather than a performance issue, it's a correctness/safety/semantics issue.

Only semi-relevant, but there's also the fact that lower level languages can auto-optimize more deeply -- but that's also more my intuition (would love to get learnt if I'm wrong).

For example, I'd expect that Rust (or rustc I guess) can auto-vectorize more than Node/Deno/etc.

Curious what you mean by "most" (I'm agnostic/unlearned on the statistics tbh). I "feel" like it doesn't happen too often when it's not either already low-level or the supposed extra performance is likely worth it.

Like, I can't imagine most people using Javascript would want to rewrite in Rust without some decent reason.

Conclusion noted: nuke the whales before they nuke us.

(/s)

I mean, the QKV part of transformers is like an "ASIC" ... well, for an (approximate) lookup table.

(also important to note that NNs/LLMs operate on... abstract vectors, not "language" -- not relevant as a response to your post though).

Everything you're saying is plausibly possible in the absurdly large search space of all possible scenarios. The author's premise, however, is rooted in the specific scenario they lay out, with historical supporting examples which you can look into. Even then, the premise before all that was essentially: Redlock does not do what people might expect of a distributed lock. Btw I do have responses to your questions, but often times in these sorts of discussions, I find that there can always be an objection to an objection to ... etc. The "sense" (or flavor) in this case is that "we are taking a complex topic too lightly". In fact, I should probably continue reading the author's book (DDIA) at some point...

The rack could go down from the point of view of the storage service, but the machine/VM itself could be perfectly fine.

What if the rack goes down? But I think the author is saying a similar thing to you. The fenced token is essentially asserting that the client will no longer rely on the lock, even if it tries to. The difference is the service doesn't need any acknowledgement, no permission needed to simlly deny the client later.

This isn't a mutex, but the distributed equivalent of one. The storage service is the one who invalidates the lock on their side. The client won't detect its own issues without additional guarantees not given (supposedly) by Redlock.

thank you for the education :) this is so good!

Can anecdotally confirm as a data point myself along with some people I know.

Certainly, I can be viewed as stereotypically successful by my buddies. Certainly, because the mask we wear should be strong and never come off. But in my nights alone, or even in my therapy sessions, who could possibly see that 'me'?

I have come to realize, though, that most people who didn't suffer much, refuse (implicitly, not explicitly) to empathize with those who have -- perhaps even I may be included in this group which 'refuse implicitly' at time. It's not that big of a deal, though. (Generalized) Empathy is a difficult skill to train.

I'm not sure if the maths are related, but there is a different (though similar) idea in gravitational slingshots. It's not the same because it simply uses "normal" linear momentum. But I also only read the abstract, lol.

Very interesting! I'd love to see discussion about how game theory/mechanism design could help with "marketplaces" like this!

something something wings are air-legs

Dang! I was gonna (lol) write an HM-in-python post, also via my experiences at Recurse Center! In other news, great to see another Recurser :)

Same, except the last point, I feel like I've seen that pattern multiple times. And it's not like it's expensive to do (presumably unless you get sued or something).