HNHacker News
TopNewBestAskShowJobs

mustache_kimono

3,844 karma · joined January 24, 2018

submissionscomments
mustache_kimono··on C Is Not a Low-Level Language (2018)
> C being a low-level language is not a myth if, as normal people do, you consider assembly languages to be low-level languages.

I'd suggest you're holding that stick too tight!

> If you want to argue that there are no low-level languages when it comes to programming modern CPUs, you are free to do so, but that is a different argument.

Not if you read the whole article?

> And it is arguably not a fruitful one because then the terminology loses all meaning. Okay, you've defined assembly as a high-level language.

Again, what if the terminology isn't very important? I'd argue the distinction between high and low level languages is not an important distinction, because it is so crude. For example, C/C++/Rust have all been described as high level languages at one time or another. C people seem to be the only ones that take real offense to this.

mustache_kimono··on C Is Not a Low-Level Language (2018)
> nonetheless intentionally misleads readers

I'm really not certain that's the idea, and it certainly does not feel very charitable. Perhaps you are holding on a little too tightly to this high vs. low level distinction (the simple mental model being attacked)? I compared the author's argument to a reductio. A reductio is not intentionally misleading?

This paper reminds me of "It's Time for Operating Systems to Rediscover Hardware". See: https://www.youtube.com/watch?v=36myc8wQhLo

There, an argument is made that our simple model of "the machine" is also wrong. There, the speaker points out much of the software that is running on our complex SoCs, with multiple cores, is firmware. To which, I'd imagine you might argue: "But that firmware is not the OS?! This talk is misleading!"

> If the author knew this, and his central premise were that no low-level programming language existed anymore, the article would be titled differently and he wouldn't be making the arguments against C specifically.

I am not sure. I believe the reason C is targeted specifically is because C communities are where this myth, and its religiosity (!), is the strongest.

> But this is essentially packaged as clickbait

I'd agree that the article is provocative, but it would seem to have good reason to be. Lots and lots of people think both C and our processors must work one way. That there is or was some level of naturalism/determinism at play. The author is simply pointing out -- not so much.

> After writing out my charitable interpretation of the author's capabilities, this interpretation only leaves me more disgusted with the article as a writing output.

Yes, it was designed to make you mad. But if you were forced to write a rebuttal to the entire article, from a charitable POV, I think you'd see there is some value to the reader in realizing this tight coupling (C and processor design) is not a necessary condition.

mustache_kimono··on C Is Not a Low-Level Language (2018)
> The article explicitly states

Again -- I think your impression is the result of the contentious tone of the article. Yes, the article explicitly states:

    "Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise’s computer at the other. Low-level languages are “close to the metal,” whereas high-level languages are closer to how humans think."
But then spends the rest of the article debunking this commonly held notion, specifically and explicitly re: C, but also implicitly re: assembly.

See the very next section "FAST PDP-11 EMULATORS"

    "The root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11. This is essential because it allows C programmers to continue in the belief that their language is close to the underlying hardware."
The author obviously knows that assembly suffers from the same abstraction penalty. The author is saying, because C and processor design has been so tightly intertwined, we cannot program "close to the metal" because "the machine" is actually a very fast PDP-11 emulator.

See also the section "IMAGINING A NON-C PROCESSOR", where the author explicitly discusses alternative processor designs (which would of course require new assembly languages!).

The author is actually trying something like a reductio on your mental model. When the author states "Think of programming languages as belonging on a continuum", the author is really saying "This is everyone's impression, but ... when you look a little deeper you see the cracks (which are actually contradictions)."

mustache_kimono··on C Is Not a Low-Level Language (2018)
> You could attempt to make the claim that assembly is no longer a low-level language, but the article explicitly does not do this, instead listing assembly as the low-level extreme that C is being compared against.

The article mentions assembly once. But it's not an argument about how assembly is "low level" and C isn't, although it may sound like that, upon a first reading, given the article's contentious tone.

The article is really an argument about how C programmers believe, and constantly state, that they are programming "close to the metal", but what they are really programming is a very fast PDP-11 emulator with lots of implicit behavior.

Implicit behavior like speculative execution and asynchronous execution and lots and lots of caching.

mustache_kimono··on Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
Wow, yeah, really hate this.
mustache_kimono··on Bun's experimental Rust rewrite hits 99.8% test compatibility on Linux x64 glibc
> Now it needs to be put into shape so that all the unsafe blocks are eliminated

All the unsafe seems to be FFI?

https://github.com/search?q=repo%3Aoven-sh%2Fbun+unsafe+lang...

> and the code is turned into maintainable, readable, reasonably idiomatic Rust. I wonder how long is it going to take.

This isn't a c2rust rewrite?

mustache_kimono··on Nobody ever got fired for using a struct
> Why not use a struct of arrays?

I would assume because then the shape of the data would be too different? SOAs is super effective when it suits the shape of the data. Here, the difference would be the difference between an OLTP and OLAP DB. And you wouldn't use an OLAP for an OLTP workload?

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity
> The two obvious examples

Appreciate this rincebrain. Know that you know better than most and this certainly covers my 2nd point. I don't imagine these cases cover my first point though? These are not bugs of the type a fsck would catch?

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity
> Two examples that I can find

I think you may be misreading my point above. I am not arguing ZFS doesn't have bugs. That's nuts. I am arguing that the bug the parent says he has would be an extraordinary bug.

This is not just a bug that a scrub wouldn't find, but also it is a bug which an fsck would find. And it is not just a bug in the spacemaps or other metadata, but the parent's claim is this is a bug which a scrub, which is just a read, wouldn't see, but a subsequent read would reveal.

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity
> There's been several instances.

I think you're missing the 2nd feature to the parent's point that I take issue with, which is this is not just a bug that a scrub wouldn't find, but it must also be a bug which an fsck would find.

The parent's point is -- ZFS should have an fsck tool because an fsck does something ZFS cannot do by other means. I disagree. Yes, ZFS has bugs like any filesystem. However, I'm not sure an fsck tool would make that situation better?

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity
> Scrubs check hashes, not structure.

How is the structure not valid here? Can you explain to us how an fsck would discover this bug (show an example where an fsck fixed a similar bug) but ZFS could never? The point I take contention with is that missing an fsck is a problem for ZFS, so more specifically can you answer my 4th Q:

>> 4) If so, wouldn't this just be a bug, like (a bug in) fsck, not some fundamental limitation of the system?

So -- is it possible an fsck might discover an inconsistency ZFS couldn't? Sure. Would this be a fundamental flaw of ZFS, which requires an fsck, instead of merely a bug? I'm less sure.

You do seem to at least understand my general contention with the parent's point. However, the parent is also making a specific claim about a bug which would be extraordinary. Parent's claim is this is a bug which a scrub, which is just a read, wouldn't see, but a subsequent read would reveal.

So -- is it possible an fsck might discover this specific kind of extraordinary bug in ZFS, after a scrub had already read back the data? Of that I'm highly dubious.

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity
> Imagine that a directory ZAP has an entry that points to a bogus object ID. That would be an example. The ZAP block is intact but its content is inconsistent.

The above is interesting and fair enough, but a few points:

First, I'm not sure that makes what seems to be the parent's point -- that scrub is an inadequate replacement for an fsck.

Second, I'm really unsure if your case is the situation the parent is referring to. Parent seems to be indicating actual data loss is occurring. Not leaking objects or space or bogus object IDs. Parent seems to be saying she/he scrubs with no errors and then when she/he tries to read back a file, oops, ZFS can't.

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity
> Imagine a race condition that writes a file node where a directory node should be. You have a valid object with a valid checksum, but it's hooked into the wrong place in your data structure.

A few things: 1) Is this an actual ZFS issue you encountered or is this a hypothetical? 2) And -- you don't imagine this would be discovered during a scrub? Why not? 3) But -- you do imagine it would be discovered and repaired by an fsck instead? Why so? 4) If so, wouldn't this just be a bug, like a fsck, not some fundamental limitation of the system?

FWIW I've never seen anything like this. I have seen Linux plus a flaky ALPM implementation drop reads and writes. I have seen ZFS notice at the very same moment when the power dropped via errors in `zpool status`. I do wonder if ext4's fsck or XFS's fsck does the same when someone who didn't know any better (like me!) sets the power management policy to "min_power" or "med_power_with_dipm".

mustache_kimono··on Understanding ZFS Scrubs and Data Integrity

    "Scrubs differ significantly from traditional filesystem checks. Tools such as fsck or chkdsk examine logical structures and attempt to repair inconsistencies related to directory trees, allocation maps, reference counts, and other metadata relationships. ZFS does not need to perform these operations during normal scrubs because its transactional design ensures metadata consistency. Every transaction group moves the filesystem from one valid state to another. The scrub verifies the correctness of the data and metadata at the block level, not logical relationships."
> ZFS scrubs do not check filesystem objects for correctness and consistency; it only checks that they have the expected checksum and so have not become corrupted due to disk errors or other problems

A scrub literally reads the object from disk. And, for each block, the checksums are read up the tree. The object is therefore guaranteed to be correct and consistent at least re: the tree of blocks written.

> Unfortunately it's possible for ZFS bugs and issues to give you filesystem objects that have problems

Can you give a more concrete example of what you mean? It sounds like you have some experience with ZFS, but "ZFS doesn't have an fsck" is also some truly ancient FUD, so you will forgive my skepticism.

I'm willing to believe that you request an object and ZFS cannot return that object because of ... a checksum error or a read error in a single disk configuration, but what I have never seen is a scrub that indicates everything is fine, and then reads which don't return an object (because scrubs are just reads themselves?).

Now, are things like pool metadata corruption possible in ZFS? Yes, certainly. I'm just not sure fsck would or could help you out of the same jam if you were using XFS or ext4. AFAIK fsck may repair inconsistencies but I'm not sure it can repair metadata any better than ZFS can?

mustache_kimono··on Read_once(), Write_once(), but Not for Rust
> That will also put it on the unfortunate position of being the place that breaks every time somebody adds a bug to the C code.

Can someone explain charitably what the poster is getting at? To me, the above makes zero sense. If the Rust code is what is implemented correctly, and has the well-defined semantics, then, when the C code breaks, it's obviously the C code's problem?

mustache_kimono··on Native ZFS VDEV for Object Storage (OpenZFS Summit)
> Why would I use it for s3?

You have it the wrong way around. Here, ZFS uses many small S3 objects as the storage substrate, rather than physical disks. The value proposition is that this should be definitely cheaper and perhaps more durable than EBS.

See s3backer, a FUSE implementation of similar: https://github.com/archiecobbs/s3backer

See prior in kernel ZFS work by Delphix which AFAIK was closed by Delphix management: https://www.youtube.com/watch?v=opW9KhjOQ3Q

BTW this appears to be closed too!

mustache_kimono··on Love your customers
> I think that companies like Oracle, SAP, and Broadcom begin to resemble specialized private equity firms

This is an entirely fair/accurate. I suppose what I am getting at is that these are just 2 different business models, and, the world can sustain a multitude of business models. There need not be only one (har har).

It's also fair to believe there is a moral dimension to one's own model which doesn't extract maximum value from the customer. Because IMHO "let's kick them in the dicks again" isn't an especially likable model, even if it is successful, and it's fair to avoid doing business with such people.

Imagine trying to sell your partners on doing business with Broadcom. If your core principle is "Broadcom needs to be around in 10 years", maybe the persistence/"kick them in the dicks" model is appealing, but otherwise, its fair for their competitors/Oxide to point out how awful dealing with a corporate sociopath might be.

mustache_kimono··on Love your customers
The next sentence is more defensible:

>> Certainly, these companies not endure as innovators: when coercion is your business model, innovation is not merely unnecessary but actively antithetical.

Oracle and VMware do seem like just rent seekers. I'm sure those rents do pay for plenty of nice things, but it's really hard for me to ever understand Oracle or VMware as an "innovator", beyond their initial innovations (their flagship DB, x86 virtualization).

> Oracle has endured nearly 50 years. Sun did not endure.

IMHO it's perfectly fine for companies to live well, and then be sold. AFAIAC persistence is only proof of persistence. Sun created plenty of wealth/millionaires too. And, by Bryan's lights, it did so mostly ethically. That's a good life.

mustache_kimono··on Toro: Deploy Applications as Unikernels
> Page management isn't really a thing we can do well "in user space".

But it is the thing most high performance OLTP DBMSs, most of us are aware of, do? I'm also not sure your cite is relevant here. Or it is at least niche. The comparison is made to LeanStore, which is AFAICT is not feature complete, and a research prototype?

Your cite does not describe a unikernel use case, but instead in kernel helper modules. Your cite is about leveraging in kernel virtual memory for the DB buffer cache, and thus one wonders how sophisticated the VM subsystem is in most unikernels? That is -- how is this argument for unikernels? Seems as though your cite is making the opposite argument -- for a more complex relationship between the DB and the kernel, not a pared down one.

> And then, further, a DB goes and basically implements its own equivalent of a filesystem, managing its own storage. Often fighting with the OS about the semantics of fsync/durability, etc.

The fights you're describing what have thus far been the problems of ceding control of the buffer cache to the kernel, via mmap, especially re: transactional safety.

If your argument is kernels may need a bottom up redesign to make ideas like this work, I suppose that makes sense. However, again, I'm not sure that makes unikernels more of an answer here than anywhere else, though.

> I don't think it's an unreasonable mental leap for people to start thinking: "I'm by necessity [cuz cloud] in a VM. Now I'm inside an OS in a VM, and the OS is sometimes getting in my way, and I'm doing things to get around the OS... Why?"

I think that's a fair thought to have, but the problem is how it actually works in practice. As in, less code seems really enticing, the problem is what abstractions are you throwing away. If the abstraction is less memory protection, maybe this is not a good tradeoff in practice.

mustache_kimono··on Toro: Deploy Applications as Unikernels
> On performance: ... In particular I am thinking how there's a whole body of research of database buffer pool management

Why? The solution thus far has been to turn off what the kernel does, and, do those things in userspace, not move everything in the kernel? Where are these performance gains to be had?

> The Linux kernel is a general purpose utility optimizing for the entire range of "normal things" people do with their Linux machines.

Yeah, like logging and debugging. Perhaps you say: "Oh we just add that logging and debugging to the blob we run". Well isn't that now another thing that can take down the system, when before it was a separate process?

> That and startup times, big world of difference.

Perhaps in this very narrow instance, this is useful, but what is it useful for? Can't Linux or another OS be optimized for this use case without having to throw the baby out with the bathwater? Can't one snapshot a Firecracker VM and reach even faster startup times?

> On security, I don't think it's unreasonable or pure "security theatre" to go removing an attack surface entirely

Isn't perhaps the most serious problem removing any and all protection domains? Like between apps and the kernel and between the apps themselves?

I mean -- sure maybe remove the filesystem, but isn't no memory protection what makes it a unikernel? And, even then, a filesystem is usually a useful abstraction! When have I found myself wanting less filesystem? Usually, I want more -- like ZFS.

This is all just to say -- you're right -- there may be a use case for such systems, but no one has really adequately described what that actually is, and therefore this feels like systems autoeroticism.

mustache_kimono··on Toro: Deploy Applications as Unikernels
> there are many of us who are very thankful for them.

Why? Can you explain, in light of the article, and for those of us who may not be familiar with qubes-mirage-firewall, why?

mustache_kimono··on Toro: Deploy Applications as Unikernels
Bryan Cantrill, "Unikernels are unfit for production". [0]

[0]: https://www.tritondatacenter.com/blog/unikernels-are-unfit-f...

mustache_kimono··on Linux Kernel Rust Code Sees Its First CVE Vulnerability
> I think Rust is more unsafe than C due to supply chain issues in the Rust ecosystem

This is such an incredibly cheap shot. First, the supply chain issues referenced have nothing to do with Rust, the language, itself. Second, Rust's build system, cargo, may have these issues, but cargo's web fetch features simply aren't used by the Linux kernel.

So -- we can have a debate about which is a better a world to live in, one with or without cargo, but it really has nothing to do with the Linux kernel security.

mustache_kimono··on Linux Kernel Rust Code Sees Its First CVE Vulnerability
> I'm more interested in the % of rust code that is marked unsafe.

I think you should less interested in % unsafe as what the unsafe is used to do, that is, it's likelihood to cause UB, etc. If it's unsafe to interface with C code, or unsafe to do a completely safe transmute, I'm not sure one should care.

mustache_kimono··on Linux Kernel Rust Code Sees Its First CVE Vulnerability
> Your sense seems more than a little unrigorous. 1/160 = 0.00625. So, several orders of magnitude fewer CVEs per line of code.

This is incorrect. Chalk it up to the flu and fever! Sorry.

0.00625 == .625%. or about twice the instance of Rust code however as stated above these are just the metric from one patch cycle.

mustache_kimono··on Linux Kernel Rust Code Sees Its First CVE Vulnerability
> Github says 0.3% of the kernel code is Rust. But even normalized to lines of code, I think counting CVEs would not measure anything meaningful.

Your sense seems more than a little unrigorous. 1/160 = 0.00625. So, several orders of magnitude fewer CVEs per line of code.

And remember this also the first Rust kernel CVE, and any fair metric would count both any new C kernel code CVEs, as well as those which have already accrued against the same C code, if comparing raw lines of code.

But taking a one week snapshot and saying Rust doesn't compare favorably to C, when Rust CVEs are 1/160, and C CVEs are 159/160 is mostly nuts.

mustache_kimono··on Linux Kernel Rust Code Sees Its First CVE Vulnerability
> So the prediction that incautious and unverified unsafe {} blocks would cause CVEs seems entirely accurate.

This is one/the first CVE caused by a mistake made using unsafe Rust. But it was revealed along with 159 new kernel CVEs found in C code.[0]

It may just be me, but it seems wildly myopic to draw conclusions about Rust, or even, unsafe Rust from one CVE. More CVEs will absolutely happen. But even true Rust haters have to recognize that tide of CVEs in kernel C code runs something like 19+ CVEs per day? What kind of case can you make that "incautious and unverified unsafe {} blocks" is worse than that?

[0]: https://social.kernel.org/notice/B1JLrtkxEBazCPQHDM

mustache_kimono··on Rust in the kernel is no longer experimental
> You don't generally need specific versions of GCC or Clang to build it I'm pretty sure.

You need a C11 compiler these days with loads of non-standard extensions. Note, for a very long time, one couldn't compile the Linux kernel with clang because it lacked this GCC specific behavior.

I'm not really sure you can turn around and say -- Oh, but now we feel differently about the C standard -- given how much is still non-standard. For instance, I don't believe Intel's C compiler will compile the kernel, etc.

mustache_kimono··on Rust in the kernel is no longer experimental
> Sometimes it's nice to know that something will run and compile reliably far into the future.

I'm not sure why you think this is a problem that Rust has? Perhaps you mean something different but the Rust project compiles available code on crate.io upon the release of a new version.[0] C compilers may imagine their new compiler code doesn't break old software, but Rust takes that extra step, so we know it won't.

Now, the Rust kernel is currently using beta and nightly features which are on track for inclusion in the stable Rust compiler. So, yes, right now compilation is tied to a specific kernel version, and may need to be updated if a feature changes. However, any C compiler used to compile the Linux kernel uses non-standard GCC extensions only recently adopted by clang. Imagine if the C standards committee chose to change the syntax/sematics of a non-standard extension. Do you not imagine the non-standard extension would also be similarly deprecated?

The issue seems to be Rust is telling you what is non-standard, and you're yelling "Look it's non-standard!". But consider that the kernel in practice is using lots of non-standard features, and should the C standard simply adopt this non-standard behavior that means likely having to changes lots of code.

[0]: https://github.com/rust-lang/crater

mustache_kimono··on Rust in the kernel is no longer experimental
>> Yes, you can use it everywhere. Is that what you consider a success?

> ... yes?

Then perhaps you and I define success differently? As I've said in other comments above, C persisting or really standing still is not what I would think of as a winning and vibrant community. And the moving embedded and kernel development from what was previously a monoculture to something more diverse could be a big win for developers. My hope is that competition from Rust makes using C better/easier/more productive, but I have my doubts as to whether it will move C to make changes.

Page 1 of 25Next →