mrlongroots

This is very clearly EAFR data, so the logical/electrical switch state. Nothing about the mechanical state of the switches has been mentioned, except a picture that shows their final state to be in the RUN position (which makes sense given the relight procedure was ongoing).

From what I understand, the relight procedure involves cycling these back to CUTOFF and then to RUN anyway. So it is not clear if they were mechanically moved from RUN to CUTOFF preceding the loss of thrust, or cycled during relight.

At the token layer an LLM can make things up, but not as part of a structured pipeline that validates an invariant that all suggestions are valid entities in the database.

Can google search hallucinate webpages?

Exactly.

To present a potential usecase: there's a ridiculous and massive backlog in the Indian judicial system. LLMs can be let loose on the entire workflow: triage cases (simple, complicated, intractable, grouped by legal principles or parties), pull up related caselaw, provide recommendations, throw more LLMs and more reasoning at unclear problems. Now you can't do this with just a desktop and chatgpt, you need a systemic pipeline of LLM-driven workflows, but doing that unlocks potentially billions of dollars of value that is otherwise elusive.

Hah thanks for sharing this, fun read!

Which, as a cpp programmer trying to pick up Rust, is honestly fine to begin with. Once you've written varying amounts of code in 5-10 programming languages, it is incredibly tedious to flip through pages trying to teach you how if conditions work and how for loops work: my brain doesn't pay attention even if I try.

This is more like: how to survive rustc as a cpp programmer which is honestly your mindframe when you start out, and it sets you up for "okay now that you speak the syntax, this is how to really think in rust terms".

This is cool!

eBPF is incredible: I think of it as flipping the relationship between you and the perf tool. Instead of the tool giving you information, it asks you "what do you want to know?".

I don't know if it's google search but I find it hard to discover blogs etc. when I'm hunting for the syntax/language etc, having documented "case studies" is super useful.

I have the same gripe. You want a canonical standard that's like "hive partitioning" but defines the range [val1, val2) as column=val1_val2. It's a trivial addition on top of Parquet.

I think there's two aspects around LLM usage:

1. Having workflows to be able to provide meaningful context quickly. Very helpful.

2. Arbitrary incantations.

I think No. 2 may provide some random amounts of value with one model and not the other, but as a practitioner you shouldn't need to worry about it long-term. Patterns models pay attention to will change over time, especially as they become more capable. No. 1 is where the value is at.

As my example as a systems grad student, I find it a lot more useful to maintain a project wiki with LLMs in the picture. It makes coordinating with human collaborators easier too, and I just copy paste the entire wiki before beginning a conversation. Any time I have a back-and-forth with an LLM about some design discussions that I want archived, I ask them to emit markdown which I then copy paste into the wiki. It's not perfectly organized but it keeps the key bits there and makes generating papers etc. that much easier.

There are DBMSes that do their own thing -- superficial chatgpt queries seem to result in a few, but I'll not mention them because I don't know much about their internals. I can think of a few reasons, mostly related to wanting to have more control over how physical media is used. I don't see those arguments made here though.

This paper may be a good read: https://dl.acm.org/doi/10.1145/3341301.3359656 -- it lays down arguments for why not to build a distributed filesystem on top of a regular local filesystem, and some of those arguments could apply to DBMS'es.

I was confused for a while about where this discussion was going, and what was the broader point. Will try to consolidate thoughts in the interest of making it clearer.

You seem unhappy with POSIX because its guarantees feel incomplete and ad hoc (they are). You like databases because their guarantees are more robust (also true). DBMS over POSIX enables all the guarantees that you like. I'd want to invoke the end-to-end systems argument here and say that this is how systems are supposed to work: POSIX is closer to the hardware, and as a result it is messier. It's the same reason TCP in-order guarantees are layered above the IP layer.

Some of your points re: how the lower layers work seem incorrect, but that doesn't matter in the interest of the big picture. The suggestion (re: microkernels) seems to be that POSIX has a privileged position in the system stack, and that somehow prevents a stronger alternative from existing. I'd say that your gripes with POSIX may be perfectly justified, but nothing prevents a DBMS from owning a complete block device, completely circumventing the filesystem. POSIX is the default, but it is not really privileged by any means.

I'm not sure how any of these negate the broader point: filesystems provide a lower-level interface to the underlying block device than a database.

To say that that's not true would require more than cherry-picking examples of where some fileystem assumption may be tenuous, it would require demonstrating how a DBMS can do better.

> Not all devices use 512 byte sectors

4K then :). Files are block-aligned, and the predominant block size changed once in 40 years from 512B to 4K.

> Hum, no. Your volume may be a sparse file on SAN system

Regardless, sequential writes will always provide better write performance than random writes. With POSIX, you control this behavior directly. With a DBMS, you control it by swapping out InnoDB for RocksDB or something.

> Thats storage domain, not application domain

It is a storage domain feature accessible to an IOPS-hungry application via a modern Linux interface like io_uring. NVMe-oF would be the networked storage interface that enables that. But this is for when you want to outperform a DBMS by 200X, aligned I/O is sufficient for 10-50X. :)

> Is file alignment on disk guaranteed, or does it depend on the file system?

I think "guaranteed" is too strong a word given the number of filesystems and flags out there, but "largely" you get aligned I/O.

> The NVMe layer is not the same as the POSIX filesystem, there is no reason we need to throw that as part of knocking the POSIX filesystem off it's privileged position.

I'd say that the POSIX filesystem lives in an ecosystem that makes leveraging NVMe layer characteristics a viable option. More along with the next point.

> Overall you are talking about individual files, but remember what really distinguishes the filesystem is directories. Other database, even relational ones, can have binary blob "leaf data" with the properties you speak about.

I think regardless of how you use a database, your interface is declarative. You always say "update this row" vs "fseek to offset 1048496 and fwrite 128 bytes and then fsync the page cache". Something needs to do the translation from "update this row" to the latter, and that layer will always be closer to hardware.

Some examples off the top of my head:

- You can reason about block offsets. If your writes are 512B-aligned, you can be ensured minimal write amplification.

- If your writes are append-only, log-structured, that makes SSD compaction a lot more straightforward

- No caching guarantees by default. Again, even SSDs cache writes. Block writes are not atomic even with SSDs. The only way to guarantee atomicity is via write-ahead logs.

- The NVMe layer exposes async submission/completion queues, to control the io_depth the device is subjected to, which is essential to get max perf from modern NVMe SSDs. Although you need to use the right interface to leverage it (libaio/io_uring/SPDK).

> "Likely the idea that filesystems should run as userspace / unprivileged (or at least limited privilege) processes which would make them, ultimately, indistinguishable from a form of database engine."

"Userspace vs not" is a different argument from "consistency vs not" or "atomicity vs not" or "POSIX vs not". Someone still needs to solve that problem. Sure instead of SQLite over POSIX you could implement POSIX over SQLite over raw blocks. But you haven't gained anything meaningful.

> Persistent file systems are essentially key-value stores

I think this is reductive enough to be equivalent to "a key-value store is a thin wrapper over the block abstraction, as it already provides a key-value interface, which is just a thin layer over taking a magnet and pointing it at an offset".

Persistent filesystems can be built over key-value stores. This is especially common in distributed filesystems. But they also circumvent a key-value abstraction entirely.

> IMO a big problem with POSIX filesystems is the lack of atomicity

Atomicity requires write-ahead logging + flushing a cache. I fail to see why this needs to be mandatory, when it can be effectively implemented at a higher layer.

> This and a complete lack of consistent networked API

A consistent networked API would require you to hit the metadata server for every operation. No caching. Your system would grind to a halt.

Finally, nothing in the POSIX spec prohibits an atomic filesystem or consistency guarantees. It is just that no one wants to implement these things that way because it overprovisions for one property at the expense of others.

Thoughts:

1. Distributed filesystems do often use databases for metadata (FoundationDB for 3FS being a recent example)

2. Using a B+ tree for metadata is not much different from having a sorted index

3. Filesystems are a common enough usecase that skipping the abstraction complexity to co-optimize the stack is warranted

To reframe your takeaway: you want to benchmark the "system" and see how capable it is. The boundaries of the system are somewhat arbitrary: is it "AI + web" or "only AI", and it is not about fairness as much as about "what do you, the evaluator, want to know".

Exactly this.

For me, the C/C++ language pack stopped working overnight with Cursor. This was clearly because of commercial concerns about derivative IDEs fairly and squarely gaining traction over the original product. But it broke my workflow a couple hours before a meeting.

I use neovim with LSPs and this is unimaginable in my world. I have started using IDEs only because the productivity gains from better LLM integration are undeniable. Sure I moved to clangd in Cursor and it was all fine, but the IDE actively pushes you to install Microsoft extensions, that can be yanked off whenever some Msft PM decides "oh we didn't actually want our competitors to be making money".

LLVM/GCC/Neovim/Apache projects are open-source. Anything that is "open-source until it is not" is not open source, and this perfectly describes VSCode today.

Now we are getting into the definition of what a parallel filesystem is.

In my book, a parallel filesystem is not just pooling together a bunch of nodes, but something that can actually support the synchronized accesses needed by a parallel workload. So not just decoupling between data and metadata, but scaling out of the metadata layer as well.

That and a hierarchical namespace (I could be sold on compromising some POSIX compliance for performance reasons, but it has to fundamentally be a hierarchical namespace with similar semantics). So object stores would not qualify.

Deepseek just used FoundationDB to build a parallel filesystem. Parallel filesystem are a big deal -- their number, including proprietary ones, is probably in single digits.

I think summarization quality can only be a subjective criterion measured using user studies and things like that.

The task itself is not very well-defined. You want a lossy representation that preserves the key points -- this may require context that the model does not have. For technical/legal text, seemingly innocuous words can be very load-bearing, and their removal can completely change the semantics of the text, but achieving this reliably requires complete context and reasoning.

> It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

It will be massive for research labs. Most academics have to jump through a lot of hoops to get to play with not just CUDA, but also GPUDirect/RDMA/Infiniband etc. If you get older/donated hardware, you may have a large cluster but not newer features.

Yes, that is sensible.

While India has a moral claim to these artifacts, it will take another decade or two of growth before there is adequate bureaucratic bandwidth to provision the necessary infra needed to maintain them at home. Until then, the status quo works.

Tip: copy-paste our entire exchange as a Gemini prompt and ask "who's correct?".

(I did that yesterday and was very satisfied :) )

I concede that the specifics of what I said were wrong, but the larger point was not.

If you buy a single DGX H100 rack and run LINPACK, you automatically get TOP500-grade numbers. Infiniband is a solid product, if not the best commercial offering for AI/ML, but no one buys it for an HPC cluster separately from the DGX boxes.

> Furthermore, if you use RDMA over Ethernet, you are using infiniband at a low level. RoCE which enables it was originally called Ethernet over infiniband.

This is wrong.

RDMA over Ethernet is... RDMA over Ethernet. There is no Infiniband involved.

RoCE was motivated by supporting RDMA, which was then an IB-only feature, over regular Ethernet. The user-level APIs are the same (verbs), but the underlying architecture is all different --- it is traditional Ethernet with link-level flow control to make it lossless (pause frames).

Sorry to hound you for the third time, but this is wrong:

> Infiniband is extremely popular in the HPC space

Not anymore. There used to be Cray Aries/GNI, psm/psm2, and now there's Slingshot, the new Cornelis stuff etc. There's almost no Infiniband now.

Or they can develop a RoCEv2 that works (which is basically what UltraEthernet is).

Infiniband is not fun - it's a special snowflake of an interconnect that sits parallel to the rest of your datacenter network, and can not really run a standard TCP/IP codebase (yeah IPoIB is a thing but still). Do the Nvidia boxes really need a scale-out IP network as well as an Infiniband network?

Plus the spec is old. Packet spraying and trimming, better ordering guarantees, queue pair scalability... a whole bunch of enhancements have been incorporated into UE all the while being compatible with regular Ethernet.

Qlogic was never really an Infiniband vendor --- their qib driver is still in the Linux codebase and essentially emulates verbs on top of a messaging-based design.

> Infiniband is an industry standard

Infiniband is not an industry standard lol.

Maybe it used to be, but it definitely is not anymore. Most Infiniband vendors are dead. The only product from those days that endures is Cornelis' Omnipath, and even that only emulated the Infiniband API back with its first gen, and then evolved to be its own thing.

At this point, Infiniband is as good as a proprietary interconnect only sold by Nvidia/Mellanox.

It's surprising that Chinese companies have not stepped into fill the printer gap. For most places where American companies stagnated, Asian ones (cars, renewables, sub-$500 smartphones) came in and ate their lunch.

To offer a different perspective, I think this is lazy criticism. There is a fundamental tradeoff between the "sharding strategy" of a social network and user experience. Both bluesky and mastodon are reasonably pareto-optimal given this tradeoff.

Mastodon goes for real "decentralization" with tiny shards of the "content tier", which means:

1. There is a lot of intra-shard bandwidth but relatively limited inter-shard bandwidth. You can create a truly seamless global experience, with global search, recommendations etc.

2. There is no uniformity in user experience (say moderation-wise) as each shard is essentially a personal fiefdom.

Bluesky instead decouples the storage layer from the caching + serving layer. This allows data portability but still allows a truly unified view of the social network.

Users are free to choose between the two approaches and most overwhelmingly choose Bluesky. But it is not very thoughtful to criticize a pareto-optimal design point for compromising one property to gain another.