Hacker News Reader

Comment by formerly_proven | Hacker News Reader

"Bring your own datastructures" is likely one of the C carry-overs for Hare. If you think about many "classic" C applications, a lot of them basically only exist around/for one or a few data structures (e.g. most servers, many utilities etc.). In that context BYODS is somewhat defensible: If you don't care to implement the data structure, why does your application exist in the first place?

But that's not "modern" applications, which often deal with a ton of data structures. C applications that do so usually have a few "template headers" lying around (indeed, certain operating systems ship with them), though one of the most frequently used data structures in C is of course the intrusive linked list. Which is not a good data structure to use in most cases. Why is it used so much in C? Because it is easiest to implement. Why does that matter? Because C makes implementing generic data structures tedious. Is it a good idea to replicate that model?

qed.

View on HN

Rust and Go seemed to initially want to target C programmers [1], but ended up capturing users from higher level languages – even Python and such – who wanted better performance and more control, because they had so many affordances.

Hare clearly isn't like that: it's _actually_ aimed at (FOSS) C programmers, almost stubbornly so, and isn't going to appeal to many others. But for those people I can see C-with-tagged-unions being an improvement, and there's value in finding the minimal diff that makes a language better.

[1] Remember when Go called itself a "systems programming language"? Neither community seems to use this phrase any more, though.

Rust still considers itself a “systems programming language” . They are moving Rust into the Linux kernel and android drivers. It doesn’t get any more “systems programming” then that. But you are correct about go, it doesn’t give the programmer enough control to be used at that level.

> But you are correct about go, it doesn’t give the programmer enough control to be used at that level.

What I like about go is that I can go full unsafe.Pointer if i want to, and do whatever I want.

The other thing I really like, is that they did such a good job discouraging it that i frequently hear people complain about go having pointers but not even letting you have fun with them.

The problem isn't that go doesn't give you enough control, the problem is that it's garbage collected. And you'd need to work around the GC. It's doable and people have done it though. Good idea? Maybe not.

Still waiting for someone to make "go but with ownership and borrow checker"

I think that being garbage collected is exactly the control problem in question.

I'm no a systems programmer unless you count hobby microcontroller projects, but my understanding is that, when a systems programmer is talking about control, they tend to be talking first and foremost about keeping tight limits on memory usage. Pointers and suchlike just happen to be key tools that help you do that.

One doesn't prevent the other,

https://www.withsecure.com/en/solutions/innovative-security-...

> Still waiting for someone to make "go but with ownership and borrow checker"

That would have interested me more a few years ago, but I'm deeply invested in Rust these days.

Is that all you'd change about Go? One of the reasons I moved was the error handling and lack of generics. Go has fixed the latter, but I can't stand not having Result<> and Option<> these days. The '?' is awesome too.

> Is that all you'd change about Go?

No, go has plenty of warts, like every language (ever formatted a date in go?). I just don't like kitchen sink languages.

> “go but with ownership and borrow checker”

Wouldn’t this be just a lesser version of rust then? If someone makes a new language and wants it to catch on, it needs new ideas. Not just nice simple syntax mixed with something someone else is already doing.

> Wouldn’t this be just a lesser version of rust then?

That depends if you consider C++ an upgrade over C :)

Many flamewars have been had on this topic and i think it's just a matter of personal taste.

I'm firmly in the less is more camp.

Hilarious that you think that you could just add a borrowing mechanism + ownership and still be in the so-called less is more camp. That language would be looked down upon by the so-called less is more folks because of the “straightjacket” it would force on the programmer.

You can't even call a non-Go library without switching to cgo, which contradictorily basically everyone says not to use.

Leaving aside the GC, until Go has a real FFI it's hard to to imagine it being a really great fit for systems programming when the systems are all not written in Go.

Go is not a traditional systems programming language and calling it one is misleading at best and maybe even borderline dishonest. Two big problems are the lack of control of memory layout and the lack of some form that compiles down to a computed goto. It’s just not possible to write performant systems code without those.

I’m not bashing Go though. I appreciate the focus on readability among other things. It’s a fine language for the use cases it was actually designed for, like pushing petabytes of ad data to serving clusters all around the world with acceptable latency and reliability.

Yes I agree it's a nice language. But the FFI situation is mildly infuriating.

https://www.withsecure.com/en/solutions/innovative-security-...

I guess the Genera, Xerox PARC, ETHZ, Microsoft folks are borderline dishonest.

I would say that they evidently felt the need to write a new compiler and runtime, which appears to be a species of agreement with my thesis.

Apparently writing compilers isn't systems programming....

In my prior comments above and in all the following I use "language" in the typical way, which is to say referring to not just the syntax but the semantics of the standard toolchain and runtime as well. I wanted to clarify that since perhaps there is some confusion there. So when I talk about Go I'm talking about what I get here[1] as is everyone else who isn't explicitly specifying some other implementation.

Writing compilers is not systems programming in the sense that it requires a systems programming language, no. One could easily write a C compiler in Ruby, but we don't consider Ruby to be a systems programming language.

Thus, obviously, Go, despite not being a systems programming language, could be used to write the compiler for a systems programming language. I guess that is what Tamago is? I'm not going to read through the source to find out and the web page you linked is boring marketing copy.

[1] https://go.dev/dl/

Go compiler toolchain is written in Go.

TamaGo is a bare metal runtime for unikernels, whose main use is a commercial product for secure USB keys, sold by F-Secure.

Back on my youth, writing compilers was considered systems programming, in parity with kernel drivers, how things change.

I guess writing userspace drivers in hybrid kernels is also not considered systems programming.

You can use the same FFI as K&R C, write a couple of Assembly helpers.

Any links to help me understand what you mean about the K&R C FFI? (Not easy to google for)

Possibly related, building against foreign objects and manually setting up the FFI call: https://words.filippo.io/rustgo/.

Easy, K&R C was rather limited, inline Assembly wasn't yet a common extension, rather you would use the external assembler, and link both object files together.

What was good for C while it was gaining adoption, surely is good enough for Go.

Gotcha, that is along the same lines as the link I shared.

For example,

https://en.m.wikipedia.org/wiki/Small-C

https://github.com/trcwm/smallc_v1

> there's value in finding the minimal diff that makes a language better.

OTOH, if the differences are minimal, then the benefits are likely to be correspondingly minimal. And either way, the compiler/interpreter for a relatively new language will by definition be much less battle-tested.

Practically speaking, I think if a new language is going to compete with C, it needs to offer more than a few incremental improvements.

Mostly because writing compilers, linkers, GC implementations, GPU debuggers, OS emulation layers, distributed orchestration systems, unikernels, isn't considered systems programming by the crowd that is allowed to judge it.

No. Rust always wanted to appeal to C++ programmers moreso than C programmers.

"Because C makes implementing generic data structures tedious. Is it a good idea to replicate that model?"

A lot of so called "modern" languages make a choice for you. Here is a hash map, over there is a class, have a tuple, set, etc. And they all have a specific way to be used.

In implementing all that, many languages lose on flexibility while gaining general utility. Hare seems to go in the opposite direction, one that makes programmers think hard about the solutions to their problems.

And that is, IMHO, a good thing.

I think the main thing is that e.g. python allows you to reimplement all of its most basic data structures in python if you so desire. There's no real way to reimplement e.g. an array in C. The result being that python could still be useful if it wouldn't come with a bunch of data structures built-in.

BYOD combined with "we don't need a package manager" sounds like a particularly bad combination. I understand Rust's "small stdlib but great package ecosystem", but whats the appeal of "anemic stdlib plus no package ecosystem"?

Maybe I would come up with something like this if I only ever worked on Linux software using C, but outside those niches these choices seem weird.

> but whats the appeal of "anemic stdlib plus no package ecosystem"?

You can declare your language feature complete sooner

Correct me if I'm wrong, but Hare doesn't say "we need no package manager" but instead "we don't need a new package manager, use the one your OS provides", at least from how I understand it.

That strikes me as another C-ism that didn't deserve to be carried forward into a 21st century language.

There's a reason we call it "dependency hell" and not "happy fun fussing with dependency version conflicts among projects that are completely unrelated except that you happen to be using the same computer to work on them time."

The state of the art in package management could be considered Nix and Guix and those are both operating system level package managers and thats why they are better than npm or pypi or cargo, as the aforementioned language specific package managers _dont_ handle dependency hell as native dependencies exist. they all have the same problem as 'they work on my computer' and also encourage this ecosystem of thousands of micropackages, which in a thread so focused on security seems a little ironic given there is no way to guarantee all of those dependencies are made by good actors without a lot of vetting that just is not happening.

I've never had a "works on my machine" bug when dependencies are managed per project. The only time I have ever had them is with global/os-level dependencies. And I've used cargo/npm a lot in the last few years.

I do agree on the micropackages.

What I keep daydreaming of is a package manager that resolves and download packages, but doesn't automatically grab transitive dependencies. I don't even want it saying, "Hey, we need to grab these 15 other ones, too, is that OK?" I don't want to hide the pain of huge dependency graphs like that; I want to feel it acutely. Give me an error message saying, "Oops I couldn't add FancyPackage because it depends on X, Y and Z transitive dependencies," and send me on a fetch quest. And I want the whole community around the language I'm working in to feel it acutely like that. That way we're all in the same boat, and collectively incentivized not to create the problem in the first place.

That is on the library developers to settle out. Keeping your library API stable requires the mind of a genious and discipline of a monk.

Now, keeping your language design stable and backward-compatible is a no brainer. Given that, it's possible to handle "dependency hells" simply by building from source.

Hare is designed to be ultra conservative, so you don't have to worry about dependency hells. Features are in code, not builtin into the language.

It is a win-win situation.

That's a great way to make a program work only on specific distros.

Seems it fits Hare perfectly then, as one of the explicit goals of Hare is to only support non-proprietary OSes.

Why? Proprietary OS have package managers.

How is being free related to having a specific package in the package manager?

I'd guess it does the opposite?

If I'm writing for a proprietary OS, that's probably Windows or OS X. Both of which give me a more stable target to aim at when it comes to what dependencies I can expect.

If I'm writing for an open source OS, that's probably some flavor of Linux. But which flavor? And which package manager does it use? And do they support all the dependencies I need, in the versions I need? Without pulling non-standard package repositories into the mix?

When open source software has difficulty working on proprietary OSes, I think it's usually not because of the package manager. More often it's because of something like a glibc dependency. At which point the domain of support isn't really "open source OSes", it's usually something more like "glibc-based linux distributions."

How so?

Because you can't guarantee that every distro's package manager will have the packages you need.

Build them from source, then.

Hare is a systems programming language. What kind of a systems programmer doesn't know how to build from source?

So the solution is just vendoring? Vendoring is very clunky—that's why we have package managers in the first place.

Language designers have every right to avoid complex features if such a feature impede the language in any way.

Hare does the smart thing - it decouples the library distribution model from the language.

After all, it's the programmers job to make sure that distribution and packaging of his/her library is as simple as possible.

Every programmer should not be re-solving the problem of library distribution just to distribute their library. This is a solved problem; a go-style git repository model is the bare minimum that a language should have.

Having to download several dependencies from random websiees and install them is not the most fun way to spend an afternoon. Especially if half of them have cryptic build errors.

So Github is a random website now?

I think that Hare's workflow and usecase encourages having very few or no dependencies.

That means, in most cases, you should just be able to download a project's source, build and use it right away.

I question the assertion that Hare's standard library is "anemic"; it includes a pretty decent set of batteries for many use-cases:

https://docs.harelang.org

Many Hare programs will not need to have dependencies at all. We encourage a more conservative approach to dependencies than is common in many modern languages such as Cargo, PyPI, npm, Go, etc. Even dependency-heavy Hare projects will not have hundreds of dependencies, but maybe dozens at most. Think more C, not Node.

https://harelang.org/distributions/

That does look like a decent collection of functionality, but a HashMap as mentioned in the submission seems like a pretty glaring omission. I can't remember the last time I wrote a program that didn't use a Map. Even dynamic languages that tend to be quite light on types almost universally include a map type.

I can definitely see the problems with the Node "explosion of modules" approach, but if we're looking at fixing issues with C then IMO "everyone implementing their own data structures and not always with the care and attention they deserve" is right up there with things like lack of proper arrays/strings and poor null handling.

It seems like it ought to be possible to find a middle ground where dependencies are easy to find, install, update and integrate with a standardised build system, but where there is a cultural norm of being conscious of dependency size and not using micro-dependencies

Take a look at this comment where I explained why a first-class hash map is not a good fit for Hare:

https://news.ycombinator.com/item?id=31258759

Package dependency resolution is fundamentally impossible to do efficiently and often broken (for legacy reasons) in specific implementations. Programmers are terrible at semantic versioning. Community package repositories are security nightmares. Under those conditions not including a package manager seems like a reasonable decision.

Most application's bottlenecks are not their linked lists, and optimizing for anything other than your bottleneck is not the wisest use of your time. Performance is a budget, and I feel comfortable spending some of that budget on simplicity.

but maps are simpler to use (if appropriately provided by the language).

maps/sets are also fundamentally building blocks for a lot of algorthms and structures, not having them makes it WAY harder to implement them. actually potential impossible as performance can easily degrade to a point where it's unusable when you use O(n) (or worse) lookups instead of O(1) ones.

this is not a case of premature optimizations but of missing fundamental building blocks

and if you don't want genetics, ok, do it like go did initially make the map special language thingy.

Linked list has a O(N) search time, which is atrocious.

A really common optimization is to replace a linear scan on a list with a hash table. In most languages, that is a trivial step to do, which means that it gets done.

There isn't any overhead.

With hare, you just blew your complexity budget on this thing.

It's not atrocious unless you need the performance. A linked list with a hundred items which is scanned every 90 seconds or something does not really demand attention for optimization, but it will be simpler, easier to write, easier to understand, and easier to debug, and those are wins are not to be sniffed at.

And for the record, Hare does have built-in growable slices, so linked lists are pretty rare in Hare. They exist in a few niche situations -- I only ever wrote a Hare program with linked lists once.

Do you remember what the reason that you went with a linked list in that case? I’ve been learning hare and was curious about where to use them.

Also, did you use a nullable pointers to represent the next node element in the linked list? Or a tagged union of ( ptr | void)? Or something else?

Yep. I used it in a kernel to create a linked list of statically allocated data structures in a context where dynamic allocation was off the table. You can see the code here:

https://git.sr.ht/~yerinalexey/carrot/tree/master/item/conso...

Went with nullable pointer types, which is also a rarely used feature in Hare.

Cool, thanks for the link and info

For sure there's plenty of circumstances where you don't need more performance, but you can't seriously argue re-implementing another linked list is simpler, easier to write, easier to understand or easier to debug than `std::vector`/`Vec`. Multiple allocations and setting up pointers is certainly more complex than allocation/copy. It's most definitely easier to write because you're not writing it, someone else already has. It's easier to read because there's a whole lot less of it. And a single contiguous allocation is way easier to debug.

My impression, although it was difficult to be sure, is that Hare's "slices" (or maybe arrays?) are actually vectors, in at least some cases.

It seems you can grow them by appending, so they aren't like Rust's slice which is non-owning or a typical array which can't grow.

I suspect a Linked List is actually almost never the right shape for a data structure in Hare, because you should just use these vectors to do that.

The two actual rationales for Linked List in the 21st century (other than "I was writing C and I don't know what I'm doing") are: 1. Concurrency, Lock Free and sometimes even Wait Free algorithms are practical for Linked List and the other costs pale into comparison on highly contended data structures and 2. I'm doing some serious acrobatics with huge lists, I constantly perform split/ merge operations and so those being cheap is crucial. Hare doesn't have concurrency, and nobody should attempt such acrobatics in a language this dangerous.

Linked lists have generally awful cache locality. If linked lists are not the bottleneck, it's because they already aren't being used for anything where performance matters (which is generally as it should be).

Reply on news.ycombinator.com