Hacker News Reader

Comment by dividuum | Hacker News Reader

Interesting. First time I've head of Wuffs. As someone that still uses C, this in particular sounds neat:

(from https://github.com/google/wuffs#goals-and-non-goals)

""" Wuffs' goal is to produce software libraries that are as safe as Go or Rust, roughly speaking, but as fast as C, and that can be used anywhere C libraries are used.

[...] Wuffs the Library is available as transpiled C code. Other C/C++ projects can use that library without requiring the Wuffs the Language toolchain. Those projects can use Wuffs the Library like using any other third party C library. """

View on HN

The language itself is a little unreadable at first glance, but the idea of it is a very good one. sbt [1] is an amazing project for small embeddable toy programs, but using fuzzers rapidly shows how unsafe the code is, and the benchmarks in the original article show the extent of performance compromises made to make it work.

It seems like this would be an interesting approach to a lot of security programming where it involves data structures, since those have typically been a source of issues. Having a memory-safe ASN.1 parser would be really nice, considering how much difficulty that has caused in the security space.

With a lot of security programming there's a reliance on constant-time algorithms, which this may not be well-suited to, however.

[1] https://github.com/nothings/stb

*stb; sbt is something entirely different :)

All of Windows (including the NT kernel) is written using the same basic proof model that Wuffs uses, except that the constraints are specified as annotations on top of C and C++ instead of as a new programming language.

I prefer the annotation approach, TBH, but I'm glad to see people working on safety. Specifically, Wuffs (like Rust) has this problem where it tries to fill the same niche as C and C++ and improve on these older languages in specific ways, but in addition to being different in ways necessary to achieve the language's objective is also different from C and C++ in totally gratuitous ways that are basically just aesthetic differences --- for example, "type var" vs "var: type".

I'm a big believer in technical continuity. IMHO, a lot of recent language developments should have instead been extensions of C, C++, Java, or Python.

> I'm a big believer in technical continuity. IMHO, a lot of recent language developments should have instead been extensions of C, C++, Java, or Python.

If that's what you really wanted, and if we always did that, then you would probably want extensions to Perl instead of Python, since it predated it slightly and was much more popular much earlier.

Instead, I think what you're really asking for is language extensions to the things you are familiar with. Those happen to be things a lot of other people are also familiar with, but that doesn't change that the same strategy results in Perl still getting the lion's share of attention. I, for one, would be perfectly happy with that, as I love Perl, but many people would not, and possibly not for any real technical reason.

Anyone that thinks the case for Rust or D is better served by extensions to C or C++ but thinks that Python is objectively a better language than Perl should probably look deeply at exactly why they hold those views.

P.S. That's not really to say you hold those views. Consider this comment less a critique of your specific statement than slight tangent spurred by it.

Not the OP, but I do share the sentiment of improving existing unsafe languages.

The C and C++ code in many relevant stacks is not going away, and many of us won't be ever be able to use D or Rust regardless of how much we like them.

UNIX clones, Windows, .NET and Java runtime implementation, WinUI/WinRT, Android (regardless of this week's annoucement)

So Microsoft Checked C, Microsoft SAL, Visual C++/clang/gcc lifetime analysers, Frama-C, hardware memory tagging, Pluton sandbox, CHERI,... are all welcomed improvements.

Yes, I'm not against improving existing languages. I just don't think it's always sufficient, so we shouldn't accept the false dichotomy of improving existing languages or creating and promoting the use of new ones to take advantage of new ideas and technologies to their fullest.

Even if Rust were to fully supplant C and C++ for at least systems programming[1], we should hope that within a few decades it's a risk of being supplanted by something else. To want otherwise is likely to inadvertently want stagnation and/or a language that while likely still improving, is having new technologies and concepts bolted on in increasingly convoluted manner, as they were not and could not have been considered when designing the language originally.

1: I think this is unlikely, if only because I don't see one language supplanting C, I see C's market cannibalized by a few different languages, with C always retaining some use.

Yeah, that I can go along with.

> I'm a big believer in technical continuity.

So am I. D is designed to be an easy transition from C and C-With-Classes. For example, here is some code in C that was translated to D (it's part of the Digital Mars C++ compiler):

https://github.com/DigitalMars/Compiler/blob/dmc-cxx/dm/src/...

https://github.com/DigitalMars/Compiler/blob/master/dm/src/d...

They look pretty much the same. The code generated is the same. In those repositories you can also see how I translated the C versions to D with plenty of examples.

The biggest impediment is the C preprocessor. You wouldn't really want to carry that forward.

After removing dependency on the C preprocessor, most of the work is global search/replacing things like `->` to `.`.

I work on (well, mostly near) an extensible C compiler, designed so extension authors can independently create extensions, and users can import them as easily as libraries: https://github.com/melt-umn/ableC/ (and I'd love to answer any questions you have about our approach or code.)

IMO this approach hasn't taken off because maintaining compatibility with C while adding safety (or really just about any property) means implementing your own sublanguage that can't arbitrarily call C functions while maintaining your safety properties. On the other hand, C being able to call into your sublanguage easier is a benefit versus jury-rigging Cargo into your build system (in the case of Rust).

On the other hand, this approach works great for adding extensions that increase the expressive power of C with new abstractions, for example algebraic data types, C++-like templating, etc.

But the C++ spec has become quite unwieldy, so I can understand the desire to start a totally new language.

The previous post on the blog has an observation that applies to languages as well

> Software has a Peter Principle. If a piece of code is comprehensible, someone will extend it, so they can apply it to their own problem. If it’s incomprehensible, they’ll write their own code instead. Code tends to be extended to its level of incomprehensibility.

As info for others, here are the annotations,

https://docs.microsoft.com/en-us/cpp/code-quality/using-sal-...

What tool/framework do they use for annotation and are there standards like SMTlib2, but for annotating those kind of things? Can this tools verify that the annotations are correct wrt the code below or above?

https://docs.microsoft.com/en-us/cpp/code-quality/understand...

Wuffs touts its lack of memory allocation as a safety feature [0]. Rust could add a `#![forbid(alloc)]` declaration and achieve the same effect. I think this would be a good idea.

Wuffs types are not parameterized with lifetimes, so they may contain references only to items with static lifetime. This means that Wuffs cannot implement linked lists, hash tables, or other common data structures. To support those structures, Wuffs will need to either add lifetime parameters and a borrow-checker (like Rust) or add reference counting (like Rust `Arc`, C++ `shared_ptr`, Golang, Java, etc).

[0] https://github.com/google/wuffs/blob/v0.3.0-beta.1/doc/note/...

That is true, but on the other hand, Wuffles allows safe unchecked array indexing, which is a very hard problem that Rust doesn't attempt to tackle--in part because it spends most of its complexity budget on the borrow checker! I for one would be extremely interested in using Wuffles in the inner loop of many routines I currently write in Rust which are performance critical, and for which I know bounds checking is hurting performance (sometimes in ways it's hard to convince LLVM to remove), but for which I can't justify the use of `unsafe`. I would particularly love it as an embedded language, rather than one needing a full-blown application.

Generational indices allow safe unchecked array indexing in Rust.

There are many papers that cover how to do this, starting with Alexis Beissegners thesis, up to the latest Rust belt project paper on GhostCell.

There are also many libraries implementing this, so that you don't have to do it yourself.

In practice, however, like the Wuffs project says, most code doesn't care about this. You only need this for very specific types of code, but if you need it, you have it in Rust.

---

On the other hand, I've asked above what happens if you try to open a png that doesn't fit in RAM, like one of those huge space photos that are >1 Tb in size.

Firefox just handles these fine, rendering only the part of the picture that you are looking at, using level of detail, etc.

This Wuff example assumes that you have enough RAM to decode the whole image at once, which is something that widely-used libraries are not allowed to assume, because they must support from embedded systems with little RAM up to absurdly huge images of our universe.

> I would particularly love it as an embedded language, rather than one needing a full-blown application.

Given the existence of [0], I imagine this is doable.

[0]: https://docs.rs/inline-python/0.6.0/inline_python/

That inline Python is the closest thing to black magic I've seen in a while. I guess I have to find an excuse to play around with Rust some time soon, it seems like such a cool ecosystem.

If you like that sort of thing you'll enjoy following the creater on twitter. Here's another of her creations

> Did you know you can just fork() the @rustlang compiler from your proc_macro?

> Unfortunately rustc is multithreaded and fork() only forks one thread. We'll just assume the other threads weren't doing anything important.

> In today's rustc, this means we're only missing the main thread in the fork. But that one was only going to call exit() after joining our thread anyway, so not very important. It just means the process always exits with a success status. So instead we grep the output for "error".

https://mobile.twitter.com/m_ou_se/status/136884418660735795...

> Rust could add a `#![forbid(alloc)]` declaration and achieve the same effect.

Isn't that pretty much what `#![no_std]` is for? You can still use an allocator with no_std, but you have to explicitly import it.

https://docs.rust-embedded.org/book/intro/no-std.html

It seeems like Wuffs is not really designed to support dynamic memory allocation at all, which is typical for usecases like oneshot parsers.

As I understand wuffs goal is to be good enough for parsers coders decoders. It looks like they can still achieve that by sacrificing some of the nice features of other languages.

Rust already has a "mode" that forbids allocations: no-std without the alloc crate.

Goals and Non-Goals explain that wuffs is not a general purpose programming language. If all the allocations are done by code invoking the function implemented in wuffs there is no point analyzing lifetimes. For the intended usecase lifetime of the objects and their lifetimes fall into following categories:

* input buffer -> prepared by caller, must live as long as you are decoding it

* output buffer -> prepared by caller, must live as long decoder outputs to it

* fixed size decoder state -> allocated during initialization

* decoder state with size depending on input -> various compression formats formats typically include information about expected sizes, used dictionary sizes and similar so that required memory for it can be allocated ahead of time. You don't want memory allocations during the hot loop anyway. And if it turns out that content doesn't match sizes from header it's an invalid file.

IMHO if Rust had a "proper" way to transpile to C, then a "C-style subset" of Rust would be a perfect library-implementation language and something like Wuff wouldn't be needed.

Go & Java don't do reference counting. They have a GC instead.

Wrong, reference counting is a GC algorithm, and both language specifications don't state what exact algorithm is to be implemented.

In fact there are tons of GC implementations across all Java implementations, including reference counting.

For example, "An on-the-fly reference-counting garbage collector for java"

https://dl.acm.org/doi/10.1145/1111596.1111597

And here is one for Go.

https://github.com/sendilkumarn/gopurerc

MISRA-C(++), Ada/SPARK also forbid it, yet they are there powering lots of embedded systems, there is a lot of stuff that can still be implemented with such restrictions.

Isn't that stupidly limiting?

Some people consider avoiding heap allocations to be smartly limiting. ;) It’s not really that limiting either, when you learn the techniques for it.

There’s a famous list of NASA’s rules for safety-critical code that you’ll see pop up now and then, that subscribes to the “avoid heap memory allocation” philosophy. https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev...

It’s not nearly as true anymore today, but working in console games 10-20 years ago, it was very common to try to avoid dynamic memory allocation because it was viewed as slow, unsafe, and unnecessary. A common strategy was to statically pre-allocate a single block of the worst-case memory needed for a given feature, because you need to be prepared to hit the worst case without crashing, so you might as well just reserve the max and focus on reducing what the maximum possible usage is.

I once debugged a crash in a console game that was caused by someone trying to be too clever by half with their allocation and copy constructors. We had a team of a dozen people working on the crash through a weekend during crunch because the crash only occurred in a release build on the console dev kit. The cost of this one stupid bug due to using memory allocation could literally accounted for in the thousands of dollars. I basically had to write a mini one-off release-build stack-tracing sampling debugger in order to trap it. So, anyway, long story short, I’m old enough to appreciate why some people suggest that avoiding allocations is a good thing.

Depends. AFAICT it’s designed to be embedded in a c project; in that context, it’s merely saying “the wuffs part does not allocate; do that in the c code”.

Makes sense.

Seems like an honest question. "Stupidly" is a key word there because humans are, without exception, vastly too stupid to deal with the combinatorial explosion that the heap presents. This is a key goal of languages like Rust: allow the compiler to deal with the complexity for you.

There are vastly fewer interacting elements when you restrict yourself to the stack: basically logic errors and pointer arithmetic become the key concerns (and Wuffs supposedly deals with the latter). Really smart people may be able to completely reason about this space.

Developers are known for imposter syndrome, but we have an unhealthy amount of overconfidence at the same time.

Yes.

I'm not so sure about the usefulness for an image decoder.

But if you are designing safety critical realtime software for controlling a car, then you kind of want the extra reliability guarantees that avoiding the heap gives you.

After NoSQL, now we also have NoAlloc :)

You should have a look at Frama-C (https://frama-c.com/) as well for this kind of stuff.

I don't understand why he just doesn't use Rust instead. The speed difference can't be that big.

From reading the docs, one main reason is that they want the output of compilation to be plain C. The Wuffs toolchain source-to-source compiles to plain C source, which can then be compiled and linked anywhere that has a C compiler and/or an FFI to the C ABI, which is a lot of places. Wuffs verifies its safety properties entirely at compile time, so once verified, it can translate the code to completely unchecked C code that is nonetheless known to be safe.

Reply on news.ycombinator.com