The search for easier safe systems programming
sophiajt.com
sophiajt.com
Sure, some devices require using static memory allocations or are quite restricted. But a lot of other "system programming" targets far more capable machines.
...then this code didn't get the point of manual memory management, which is to minimize allocations and move them out of the hot path. Unfortunately a lot of C and C++ code is still stuck in a 1990's OOP model where each individual object is explicitly constructed and destroyed, and each construction and destruction is linked to an alloc/free call. If you do this (or reach for reference counting, which isn't much better), a GC is indeed the better choice.
https://tigerbeetle.com/blog/a-database-without-dynamic-memo...
As apposed to the language, compiler or system always making those decisions.
Absolutely not. It just requires a certain amount of planning and puts work on the callers to ensure that they prepare a suitable allocation somewhere, which may be on the stack or inside some structure or array that already exists.
The limit case is the common embedded style of statically allocating everything. Often combined with a ban on recursion and (manually or automatically enforced) limits on stack size. If you don't have a lot of bytes you can plan your usage of each of them individually.
MISRA rule 20.4 just plainly states "Dynamic heap memory allocation shall not be used." The very opposite of GC.
https://wwwfiles.iar.com/superh/guides/EW_MISRAC2004Referenc...
There's a 3x speedup sitting in a library because it's in a sea of carefully optimized functions manipulating floats, and someone made a change to extract a function that returned an array.
It was such a good Chesterton fence: change seemed straightforward and obvious, better readability. But, it had cascading effects that essentially led to 9 on-demand array allocations in the midst of about 80 LOC of pure math.
In my experience, there's hangover from two concepts:
- Android does garbage collection and Android is slower than iOS
- You can always trade CPU time for memory and vice versa (ex. we can cache function results, which uses more memory, but less CPU)
Thing is, not only has Android runtime witness several JIT and GC refactorings since Darwin was created, originally worse than Sony and Nokia's J2ME implementations regardless of Google's marketing otherwise, there are so many factors contributing to Android's perceived slowness from bad coded UIs, sloppy programing, cheapstakes OEMs with low performance chips, and so on.
On the other hand on iOS side, due to lack of GC and OS paging, when memory goes bad, applications just die.
There's a lot of things going on that make this the case.
Android was built with unrestrained multitasking; there's a lot of restraint now, but there's probably still a ton of stuff going on from unrelated processes while your foreground app is chugging along. iOS locks down background execution, and I suspect system apps have limited background execution as well. Less task switching and less cache pollution help a lot.
iOS only runs on premium phones. Even Apple's lower priced phones are premium phones. Android isn't necessarily quick on premium phones, but it can be much worse on phones designed to a $50 price point.
IMHO, A large issue is that the official UI toolkits are terribly slow. I'm more of a backend person, but I put together a terrible weather app, and first paint is very slow, even though it starts with placeholder data (or serialized data from a previous run); using an HTML view was much faster because it only makes one toolkit objeft and more capable because css layout lets you wrap text around an image nicely. Maybe there's some trick I'm not aware of to make 'native ui' not terrible on Android... but that experience helped me understand why everything is so slow to start.
However "if you do it the naive/obvious way, it performs this badly" is not a good property of a system and still probably explains why so many things are dog slow to get going.
(I doubt I'd do any better than you did myself, mind, I'm pretty sure I have even less idea what I'm doing on android than you do)
I'd be willing to bet the price of a pint that if either of us flailed around on iOS instead we'd probably end up with much better startup time even hardware-for-hardware because the easiest path for us to flail down is probably much saner than the equivalent on android.
(insert rant about APIs needing to make sure the natural path that a newbie takes is also a path that produces -good- results even if they'll still need to git gud to achieve -excellent- results)
I sincerely hope you're right. But I was following the path as best I could, so I'm just not sure how it could have gone so wrong. I do 100% agree with your assessment of my flailing though. ;)
It wasn't the only source of frustration (the weather API I was using looked good, but is actually crap, and their support blew me off when I pointed out data inconsistencies, so that's ugh). So it was easy to discard. Back to weather prediction by induction.
I wrote Java + layouts for the one big UI feature I did, then was one of the first to do Kotlin, then moved to Flutter* from then on.
In that vein: I cannot describe intensely enough how much I disliked Kotlin, and I get the vibe compose is not quite ready yet and isnt really moving the ball forward, as much as it's happy to be at general parity with SwiftUI. I really wish Flutter "won", it's such a beautiful and pleasant experience. Your job is a different job with hot reload.
* Note: that was very unofficial, im almost certainly the only person who did on Android who was writing flutter, and it was because I ended up doing design system heavy work where having a web app to show designers things became paramount, no one was expecting much work on Android proper from me
< HellKat> the clock itself is set to UTC but it has a weather facility too - if you set the GPS coordinates
< HellKat> temps right, weather however is wrong, comp says cloudy, looking outside says raining
Depends how you define "a ton of time". Good general purpose memory allocation algorithms, like the TLSF algorithm [1], guarantee a O(1) bounded response time suitable for real-time use. To your example however, if someone introduces extra computation into math-heavy hot-looping code, then that is just sloppy development as their adding extra computation. That extra computation being memory allocation is tangential.
[1] https://www.researchgate.net/publication/4080369_TLSF_A_new_...
I described observing programmers I've worked with struggled to understand that memory allocation could *ever* have a significant computation cost.
Note that GC cannot possibly promise "time-boxed pauses" in the general case; either allocations are allowed to outpace collections, in which case a longer collection pause will eventually be required to avoid memory exhaustion, or else allocations must be throttled, which just pushes an unbounded pause forward to allocation time.
ETHZ => Oberon, Oberon-2, Active Oberon, Component Pascal (via Oberon Systems, a startup spinoff out of ETHZ), Oberon-07.
DEC Olivetti => Modula-2+, Modula-3
Microsoft Research => Singularity, Midori
Genera and TI => Lisp Machines
Bell Labs => C@+ (really confusing name, one of the few sources are a couple of DDJ articles), Limbo
The way they ended up not gaining traction had more to do with management decisions, company acquisitions and hardware costs, than the technology stacks themselves.
The "C" for systems programming isn't only pure C as defined by K&R C, or ANSI/ISO C, rather C + Assembly, or compiler specific extensions for inline Assembly, pragmas, intrinsics, and keywords not part of any standard.
D and C# are two modern examples of GC based languages with similar capabilities in intrinsics.
For example, Meadow uses a C++ mikrokernel, with everything else written in C#.
Theseus is a research OS written in Rust that seems to include C purely to show how to support programs written in C (libc and a demo app).
https://github.com/theseus-os/Theseus
It contains some 1000+ lines of ASM for the early boot from BIOS, that look like they originate from https://os.phil-opp.com/minimal-rust-kernel/ -- I think UEFI boot would have been doable in pure Rust (https://github.com/rust-osdev/uefi-rs).
While Rust made Cyclone's type system more manageable for mainstream computing, everyone else is trying to combine the benefits of linear/affine type systems, with the productivity of automated resource management.
Naturally it would be interesting to see if some of those attempts can equally ping back into Rust's ongoing designs.
However that comes with performance drawbacks (or at least unpredictable/unreliable performance) which creates the need for languages like Rust. It's great to see the progress in those languages as well.
Well, in the sense that garbage collectors solve memory management. But pure functional programming in the Haskell sense doesn't really manage file handles for you, or database connections.
You could have more predictable performance in Haskell, if you didn't have to deal with mandatory laziness for some functions.
(Basically, in this hypothetical variant of Haskell, by default functions would be safe to be evaluated in any order, strict or lazy or whatever. If you want functions that need to be evaluated lazily, then you'd need to declare that; just like today you already need to declare that your function might have side effects.)
The compiler would then be free to re-order evaluation a lot more, and pick predictable, fast performance.
No, I was actually talking about the latter and not about memory management at all.
In Scala we even achieve this on the library level. (e.g. https://zio.dev/reference/resource/ which also compares the existing problematic try/catch with its own approach)
And despite Scala being strict, the GC still makes it unsuitable for many cases where you just need something like Haskell.
Not a sensible default.
You cannot ask a strict function to behave lazily. But you can ask a lazy function to behave strictly.
"The problem is the very idea of a lazy language, which imposes a ruinous semantics on types. An eager semantics supports the expected semantics of types (for example, the natural numbers are the natural numbers) and, moreover, admits definition of the lazy forms of these types using suspension types (Chapter 36). A value of a suspension type is either a value of the underlying type, or a computation of such a value that may diverge. There being no representation of the eager types in a lazy language, it follows that, eager languages are strictly more expressive than lazy languages." [1]
This paragraph is then followed by a link to Harper's "PCF by Value" supplement/errata to the above referenced PFPL, where such a statement is reinforced. [2]
[1] - https://www.cs.cmu.edu/~rwh/pfpl/supplements/commentary.pdf
You make a lazy function strict by asking for its output now rather than later.
> An eager semantics supports the expected semantics of types (for example, the natural numbers are the natural numbers).
What a (counter)example! In lazy semantics the natural numbers are the natural numbers. In eager semantics the natural numbers are stack overflow.
> moreover, admits definition of the lazy forms of these types using suspension types
This is the required modification that your library maintainer would need to make if he had produced a strict library that you wanted to use lazily. Hence "You cannot ask a strict function to behave lazily."
> There being no representation of the eager types in a lazy language
Because the lazy representation is the same as the eager representation in the source code. It's up to the caller to demand values now, or later, hence "But you can ask a lazy function to behave strictly."
That doesn't mean you can evaluate the function strictly. Have a look at this example:
f = head . map (2+)
x = f [5, undefined]
In any case, my suggestion is not to deal with arbitrary functions; but to restrict the set of untagged functions to those where any evaluation order works. 'Works' in the sense that any evaluation order takes a finite amount of time.
Huh? I suggest that when you don't annotate anything, you can only use constructs that can be evaluated in any order (including both lazy and strict or any other order). So the compiler has maximal freedom.
So your supposed impossibility doesn't come into play: we are not dealing with arbitrary functions.
If your function needs a specific evaluation order, you'd need to tag that.
To do the reverse would require uncomputation. You can't turn all the natural numbers back into the expression [1..].
That's what monads are for.
Haskell doesn't require that you use automatic management. But it does absolutely have a solution.
You might want to look into 'indexed monads', they can perhaps help with that. But they aren't a type monads (it's the other way round, vanilla monads can be seen as a trivial kind of 'indexed monads', I guess).
Yes, linear / affine types and uniqueness types can give you a lot of control over resources, while also allowing you to preserve functional semantics.
I would like to see a version of Haskell that doesn't just track IO, but also track totality. Ie unless your function is annotated with a special tag, it has to be guaranteed to return a value in finite time, ie it has to be total. If you tag it with eg 'Partial', you can rely on laziness.
That's very similar to how functions in Haskell can't do any IO, unless they are tagged with 'IO'.
(I know that Haskell doesn't see 'IO' as a tag or annotation. But it behaves like one in the sense I am using here.)
In languages for doing practical calculations, a progress dialog and a cancel button are more useful than a totality guarantee. It should be easier to make complex, long-running calculations safely cancellable.
(Still true with laziness, though it changes a bit. At some point you will ask for a value to be calculated.)
So a default constraint on functions to be total, would be sensible and helpful.
To be able to do the opposite, constrain a function or continuation to never terminate (outside of program termination), would be useful too. Albeit in much fewer contexts.
Any constraint that forces implementations to match respective design intentions is good.
It’s kind of pain when you’re not making a theoretical argument. You might still want to eliminate all divide by zero errors. Satisfying the compiler of this using a machine-checked proof might be more trouble than it’s worth, though. It might require you to use a language whose types support proofs.
That can take the form of something like `try_divide` that returns a `Maybe` that you need to pattern match on. Or it can take the form of trust-me-bro via the equivalant of UnsafePerformIO, where you know that something is safe and you want to let the compiler know, but you can't prove it to the compiler.
You can also deal with errors by returning an `Either`. That doesn't mess with any totality guarantees.
Reporting an error to a function's caller isn't really a thing in a proof. A function proves that if any input exists, its result exists. Returning an error means "an error might exist." We don't want to return that, we want to eliminate the error from our proof, so that it proves that the result exists for any input. Returning an 'Ether' type might be useful in a proof, but a 'Maybe' type seems dubious. We already assume that 'None' exists, so proving that 'None' might exist is useless.
We don't have the same concerns in proofs and practical programs.
In a practical program you want to know that you in finite time you either get one of the declared errors, or you get a positive result. Using an `Either` type allows even a relatively weak type system to be of use here.
There's no 'error' in your proof: your proof can't do anything about eg running out of disk space.
Now, using an `Either` type for things like division-by-zero is a bit of a cop-out, and a compromise foisted on us by a weak type system. And you are right that in something like Agda you would hopefully never have to stoop so low. (Though even there, you can't proof everything. Eg if your program enumerates differences between adjacent prime numbers, you can't prove that it will (or won't) run out of 2s.)
In previous post you quoted the Dhall documentation:
> The main benefit of evaluation being finite is not to eliminate long-running programs but to make them significantly less probable. In practice, you will discover that you will rarely author a configuration file that takes a long time to evaluate by accident.
This might be practical advice, but it's not theoretically sound. It's drawing a conclusion that wasn't proven. Isn't it kind of weird to write proofs without caring about the result of the proof?
You are technically right, but in practice, finite time does help a lot. To quote from the Dhall documentation at https://docs.dhall-lang.org/discussions/Safety-guarantees.ht...
> Note that a “finite amount of time” can still be very long. For example, there are some short pathological programs that take longer than the heat death of the universe to evaluate. The main benefit of evaluation being finite is not to eliminate long-running programs but to make them significantly less probable. In practice, you will discover that you will rarely author a configuration file that takes a long time to evaluate by accident.
> It should be easier to make complex, long-running calculations safely cancellable.
Just send kill 9 to your program, if you want to cancel it? That seems like an entirely separate topic? (Especially if your calculation is pure, it's always safe to cancel it.)
Go’s Context API handles cancellation and timeouts at the request level, but it has to be coded manually; a badly-behaved goroutine could still leak and consume resources long after it’s cancelled.
So it might be nice to have automatic support at the language level, to make guaranteed cancellation happen by default. This is the approach taken with structured concurrency. To preserve the invariant that a child task doesn’t outlive the parent, there needs to be a way to kill any unneeded child tasks when a function returns early.
Totality checks seem like a poor substitute when what you actually want is to prevent resource leaks.
I haven’t used Dhall, but it seems like a config language has different concerns?
I don't disagree, but...
Except for excutable sizes and maybe unpredictable performance (performance unpredictability on Haskell has the same shape of UB on C, in that it creeps in, but it's way easier to keep away), there's actually no feature missing.
You can argue for C-like speed (instead of Java-like), but the stuff on the article doesn't have it either.
People have written kernels and unikernels in Haskell. So in that sense it's definitely a systems programming language.
However, in the grand scheme of things heap vs stack allocation is minuscule. Many other factors are much more important for performance.
That said, as a sibling already pointed out, it's standard to control stack allocation with a single counter. It's kind of standard to control heap allocation with an index and a lot of book-keeping.
But you are allowed to optimize the heap until there's no difference.
Also, heap allocation is unpredictable. It is more likely to cause unexpected page faults or thread congestion (multiple threads often share the same heap so they need to synchronize access to memory book-keeping structures). Especially when it comes to kernel drivers, a page fault can lead to a deadlock, infinite recursion, or timeouts.
I'm not saying heap is always bad, not even that it's bad most of the time. But if a language doesn't at least give you the _option_ of having objects live on the stack, I wouldn't consider it a serious systems programming language.
* Carp: https://github.com/carp-lang/Carp
* Nim: https://nim-lang.org/
* Zig: https://ziglang.org/
Wow. It's like teaching C++ and starting from SFINAE. Or C# and starting from type parameter constraints.
Please think of a real-world examples when teaching stuff. I am very eager to see the program a beginner would need to write that requires: 1) references in a struct; 2) 3 separate lifetime parameters for the same struct.
I think this is a mistake, both on June's part and on Rust's. All low-level languages (by which I mean languages that offer control over all/most memory allocation) inherently suffer from low abstraction, i.e. there are fewer implementations of a particular interface or, conversely, more changes to the implementation require changes to the interface itself or to its client. This is why even though writing a program in many low-level languages can be not much more expensive than writing the program in a high-level language (one where memory management is entirely or largely automatic), costs accrue in maintenance.
This feature of low-level programming isn't inherently good or bad -- it just is, and it's a tradeoff that is implicitly taken when choosing such a language. It seems that both June and Rust try to hide it, each in their own way, Rust by adopting C++'s "zero-cost abstraction approach", which is low abstraction masquerading as high abstraction when it appears as code on the screen, and June by yielding some amount of control. But because the tradeoff of low-level programming is real and inescapable, ultimately (after some years of seeing the maintenance costs) users learn to pick the right tradeoff for their domain.
As such, languages should focus on domains that are most appropriate for the tradeoffs they force, while trying to aim for others usually backfires (as we've seen happen with C++). Given that ultimately virtually all users of a low level language will be those using it in a domain where the low-level tradeoff is appropriate -- i.e. programs in resource-constrained environments or programs requiring full and flexible control over every resource like OS kernels -- trying to hide the tradeoff in the (IMO) unattainable hope of growing the market beyond the appropriate domain, will result in disappointment due to a bad product-market fit.
Sure, it's possible that C++'s vision of broadening the scope of low-level programming was right and it's only the execution that was wrong, but I wouldn't bet on it on both theoretical (low abstraction and its impact on maintenance) and empirical (for decades, no low-level languages have shown signs of taking a significant market share from high-level languages in the applications space) grounds. Trying to erase tradeoffs that appear fundamental -- to have your cake and eat it -- has consistently proven elusive.
Various features accommodating low-level programming -- those that may have helped C++ take market share away from C but, crucially, have not helped it gain market share in application programming (over the long term) are, therefore, irrelevant to this core question. It's one thing to make a low-level language more attractive to low-level programming, and a whole other thing to make it more attractive to application programming. C++ has succeeded in the former but failed in the latter.
I would say that an higher level language provides more abstractions than a lower level one. Memory and resource management is an aspect of it, but not a requirement (C is an higher level language than asm, yet it doesn't provide any memory management other than implicit stack allocation). In any case yes, the definition is nebulous.
I would add that at the turn of the millennium C++ had a clear domination of the application space.
edit: I don't claim any specific knowledge, but, more than GC, I would say the lack of built-in networking and enterprise[1] features to integrate with web application stacks prevented C++ from taking an hold in the early Internet era (outside of large tech companies).
In the last decade or so, I think the main reason it is being displaced in the desktop space simply by virtue of not being JS.
[1] I'm not using this disparagingly
Correct, which it has since lost. My theory is that it's been due to two reasons: improvement in the performance of high-level languages due to both software and hardware advances and due to it taking several years for people to observe the cost of a low-level language, which mostly manifests not in the initial writing but in maintenance (because low abstraction means that changes to components require less local code changes). It's not much more costly to write a program in C++ than it is in Java or Python, but it is much more costly to maintain over time.
> In the last decade or so, I think the main reason it is being displaced in the desktop space simply by virtue of not being JS.
It's also been displaced in the server space. But suppose the lack of built-in "enterprise" features is the cause. It's been a decade since Rust, the "modern C++", has appeared and it, too, has failed to make significant inroads into the application space (i.e. take a significant market share from what I call high level languages).
I'm not saying people should stop looking for some holy grail of low-level/application convergence, only that I would (currently) bet against it.
Re maintenance, I'm not sure that a C++ program is more costly to maintain. I suspect that C++ is over-represented in multimillion line codebases.
In any case, I think you might be right, it certainly is possible that an optimal "full stack" language (if you allow me the term) just doesn't exist. I would settle for a control-plane language that is easily embeddable and can be easily embedded in a native C or C++ (or rust, or whatever) application.
I think both have been significant, but I agree that top performance is not so much an issue for many applications (although JS is also very fast).
> I suspect that C++ is over-represented in multimillion line codebases.
You're right, but the question is compared to what? Java, C++ and C are pretty much the only languages used in multi-MLOC codebases maintained over many years (there are others, but not nearly to the same degree as those three) so they're all over-represented, but the amount of C++ and C code is significantly lower than that of Java code. I think it's due to maintenance costs (as a maintainer of a C++ codebases over my career, that's been my experience, but I try not to extrapolate so I consider this my unproven hypothesis).
High level system languages, that provide a good programming confort, while havig the tools to go under the hood, if so desired.
Being forced to deal with naked pointers and raw memory in every single line of code like C, is an anomaly only made possible due to industry's adoption of UNIX at scale.
The question I'm interested in here is not which features a low-level language should add to make it more attractive for low-level programming, but should it add features that are primarily intended to make more attractive for application programming. The declining share of low-level languages (by my definition) for application programming over the last 30 year, leads me to answer this question in the negative. This is a big difference between the approach taken by low-level languages like C++ and Rust, which try to appeal to application programming, so far unsuccessfully, and low-level languages like Zig, which don't. So far, neither Rust nor Zig have been able to gain a significant market share of low-level programming, let alone application programming, which makes judging the success of their approach hard, but C++ has clearly failed to gain significant ground in the application space despite achieving great success in the low-level space.
The reason I'm focusing on this question is that this article specifically calls out an attempt by the June language to appeal to application programmers, and I claim that C++/Rust's "zero cost abstraction" approach does the same -- it attempts to give the illusion of high abstraction (something that I believe isn't useful for low-level programmers, who make the low-abstraction tradeoff with their eyes open) without actually providing it (clients are still susceptible to internal changes in implementation).
Likewise Rust being adopted by Linux kernel, Microsoft (now the official language for Azure infrastructure, with C and C++ requiring clearance for new projects), Google on Android and Chrome/V8, it is also quite successful for low level coding, alongside the whole GCC/LLVM infrastructure, written in C++.
For example, if I deploy to Vercel or Netfly and don't want to pay for additional infrastructure for webservers, while caring for performance, my options are nodejs addons/serverless in C++/Rust, or eventually Go (which I rather not).
C++ certainly had a high market share in the application space in the early nineties, which it has since lost. The market share of low level languages in the application space overall has dropped by a lot since then and is showing not even a hint in reversal of the trend.
> Likewise Rust being adopted by Linux kernel, Microsoft (now the official language for Azure infrastructure, with C and C++ requiring clearance for new projects), Google on Android and Chrome/V8, it is also quite successful for low level coding, alongside the whole GCC/LLVM infrastructure, written in C++.
I am not doubting that a language that follows C++'s design philosophy may ultimately take a large market share from C++ or from C (although if Rust is the language to do that, its rate of adoption is alarmingly low, but that's beside the point). My point is that, despite trying, all such languages combined have not shown any success in taking a significant market share from high-level languages in the application space. Surely if the problem wasn't the philosophy but the execution, and if Rust is the language that successfully cracked the code for low-level/application convergence, we should have seen some significant change in trend over the past decade, but it's just not there. At some point, hypotheses regarding utility and value need to become very visible in the market -- which cannot be completely irrational given economic pressure and competition -- and you can try to find a trend in noisy volatility only for so long. If C++'s 40 years and Rust's 10 have failed to to prove the hypothesis that low level languages can grab a large share of the application space, maybe we can wait another 20 years and see. But I'm saying that right now there doesn't seem to be any market movement supporting that hypothesis even though I think we're long overdue to see some if the hypothesis held, and so speculation on the future success of the approach is not justified by what we see in the market.
My hypothesis for the preponderance of such speculation in forums such as HN is that commenters in such forums (as well as people writing OSes at Google) are not sufficiently aware of the software landscape overall (which is mostly growing in the "low skill" area, let's call it), of which they're making up a dropping proportion. There are 50 new Python developers for every new C++/Rust developer, and if you think you can convert a large portion of them then you haven't met them.
That's why I think that low level languages that try to accommodate application programming are misguided if they think this will help convert a significant portion of application programmers in a market that's overall becoming less skilled, not more (I think this approach harms their adoption rather than helps it). I believe that the low-level/application convergence a lost cause, but even if it isn't, then Rust certainly isn't the language to have cracked this problem. I would say that the only hope for "high skill" languages to increase their share is if the "low skill" segment is reduced by AI, which may well happen, but that's a very speculative bet.
I personally love C. I think designing a language top-down is a poor approach overall, I prefer the bottom-up approach of the C-inspired for system languages, that aim to fix C rather than this is how the world should beeee!
My own experience coding in this style [1] has been extremely reassuring.
You can generally really safely consider only the context of each process at a time, since there aren't even any function calls between processes, only data sharing.
This meant for example that I could port a PHP application that I had been coding on for years, fighting bugs all over, into a flow-based Go application in two weeks, with a perfectly development time perfectly linear to the number of processes. I just coded each processes in the pipeline one by one, tested them and continued with the next. There were never any surprises as the application grew, as the interactions between the processes are just simple data sharing which can't really cause that much trouble.
This is of course a radically different way of thinking and designing programs, but it really holds some enormous benefits.
https://github.com/rdfio/rdf2smw/blob/master/main.go#L58-L15...
Probably works quite well for systems programming, where things are either "live forever", "reallocate within some pool" (thread handles, file descriptors, etc), or "transient" (for the lifetime of a system call or similar).
Rust's ideal case are domains where allocation management is strictly controlled.
What domain's are those? Do you mean just embedded or perhaps Firefox? Please elucidate.
Everything else is a matter how much one wants to involve themselves into politics of the right way to do memory management, versus get to a compromise for more safer infrastructure.
Right, isn't that what GhostCell and its variants (QCell) are all about? This would be great if it led to a more elegant and principled implementation of that pattern, that could also end up being fully supported in Rust itself.
It may also be nice if constructs that don't cause memory accesses and only ever do the correct thing or crash on the target cpu (such as integer division or pshufb without an address operand on any intel chip ever) were not unsafe. Placing "Well, LLVM says this arithmetic operation is UB and we won't bother to fix it" and "What if one day there's an x64 chip that does something other than crash if it encounters instructions from ISA extensions it does not have?" into the same bucket as "playing with raw pointers" is a bit weird.
It's the first time I hear your second point about intrinsics, for example, and it seems like something that could be done relatively straightforwardly, but introducing "inconsistency" for the benefit of skipping a single unsafe block sounds like the kind of RFC thread I wouldn't want to be a part of.
I accept that languages that are designed collaboratively by huge numbers of people through an RFC process are basically impossible to improve.
But I think the ISA extension thing is absolutely a good idea for the simple reason that if that ever changed you would be -screwed- and forcing people to Think Really Hard about their usage if ISA extensions is probably a net win anyway.
Possibly they should be in a second "bucket of scary stuff" with a different set of hazard stickers applied, but using unsafe for all of it makes sense from the POV of 'unsafe means I am voiding my warranty inside this block' rather than anything more specific.
Please, no exceptions. No exceptions.
There was a language featured on HN recently that did a good job of making it trivial to build sum types for results/errors from disparate other sum types for results/errors. It can be made easy while retaining the idea of using a sum type for results.
1) Prefer the stack wherever possible.
2) RC.
3) Unique pointers by default unless explicitly noted otherwise.
4) Immutable arguments and returns by default unless explicitly noted otherwise.
There's a language that does this all already, and it's called Nim.
let x = new Foo()
if (y == 1) {
recycle x
}
println(x)That would be trivial to do as a runtime check, don't know if the compiler is smart enough to catch such things at compile time.
Also not sure if the idea of a 'copy count' is better than a 'generation counter' which also provides runtime protection against dangling access, e.g. see https://floooh.github.io/2018/06/17/handles-vs-pointers.html
It is the be-all and end-all that will make those scenes in Star Trek where they alter some core starship system programming in 2 minutes flat without mistakes actually plausible.
What's really lacking are safe, easy, strong typed general purpose languages that can leverage AOT compilation for high performance and static analysis etc. A language with the learning curve of Python, near C performance and strong safety guarantees of Rust. There is nothing that suggests to me such a language would not be possible. Swift and C# come closest but they are warped by their respective corporate overlords.
Floodgates will open though once class is also supported.
Lets not forget that C# only exists because Sun won the lawsuit over J++, where several C# features were born (J/Direct => P/Invoke, events, JFC => Windows Forms, COM support), and Ext-VOS (the COM next generation runtime) paper refers to J++, before COOL became C#, and J# was introduced as migration path from J++.
Ironically, after all these years, as means to keep Azure relevant for the Java community, Microsoft has become a OpenJDK contributor by acquiring jClarity, including ARM support on Windows, better escape analysis, and has their own distribution.
And VSCode plugins for Java, developed in collaboration with Red-Hat, are much better than C# Dev Kit, without requiring a Microsoft account.
The description is confusing though. I suppose they did the whole "rebrand" thing because for years everyone thought poorly of experience of writing C# in VSC while it has always been semi-decent (depending on your skills) and got better recently, especially after moving to Roslyn-based LSP from Omnisharp.
This allows for a spectrum of JVM implementations, where basically any CPU capable of running some form of Java implementation has a JDK available.
https://en.wikipedia.org/wiki/List_of_Java_virtual_machines
Likewise, most frameworks are driven by set of companies, based on industry standards, where .NET due to its culture, most Microsoft shops tend to only adopt what comes from Microsoft.
It is a long discussion problem in the .NET community that when many FOSS projects finally manage to get enterprise adoption, Microsoft ends up coming with their own solution, which said enterprises then switch to, thus killing those FOSS efforts.
Listen to big podcasts like .NET Rocks, where this tends to be often discussed with their guests.
Does Microsoft somehow prevent anyone from developing a CLR implementation for CPUs they don't support? The only reason the JVM is everywhere is because they had first-mover advantage.
How does Microsoft developing their own solution to a problem "kill" FOSS efforts, exactly? Microsoft's solutions are almost all also FOSS. Is competition some kind of inherent problem? Or is a large corporation creating and maintaining FOSS a problem? Why aren't you annoyed at those enterprises that started adopting the original FOSS projects, but didn't contribute dev time to maintaining them? Instead you're annoyed at Microsoft for creating its own FOSS and paying to maintain it for the community for free.
I just find most objections to .NET so bizarre, it's almost always knee-jerk Microsoft hate.
.NET virtual machine, MIT license: https://github.com/dotnet/runtime
SDK, MIT license: https://github.com/dotnet/sdk
You were saying?
No, the point is, who cares as long as everyone is happy with the direction?
Now granted, I don't know if it would be different with Java and Oracle. But that is the point. A significant enough change proposal for C#, supposed like above, a change proposal that goes against where Microsoft thinks the development path of C# should be, that is the important bit. So long as contributions and feature request align with Microsoft's interest, its all good, but if the community has large support for a contribution or change request that doesn't align, then it may be different.
Addition: Although arguably, this is true for most languages. Even FOSS languages have some type of leadership that those big shifting proposals will live and die with. Not intending to cast anyone in a bad light, but just a project leader and a project I am familiar with. If a proposal that was as ground shifting as the proposed above was suggested for Zig, whether or not the project does it, the decision of whether to do it may live or die depending on Andrew Kelly or his close knit team of contributors to Zig. Same could be said about Rust, the project has leadership and they could kill a popular proposal. Even outside of languages. The linux kernel, a feature or restructure may live or die by the hand of Linus Torvalds.
https://github.com/dotnet/csharplang/blob/main/meetings/2024...
https://github.com/dotnet/csharplang/blob/main/meetings/2024...
Speculation about what Microsoft may do or may not do in response to community pressure is a weird kind of criticism. I can invent all sorts of fictional boogeymen too. What if Linus has a psychotic break and wants to make the Linux syscall interface mimic the Windows NT kernel?
People would do exactly what I'm suggesting here: fork and continue development without him.
As you also say, leadership means leading, and Microsoft takes backwards compatibility very seriously, and that's why people trust them and use their offerings, sometimes instead of open source offerings.
A change like that would invalidate every tutorial and document ever written about C# over 25 years, so is it really C# anymore? I don't think anyone would make that change. It's simple to fork the MIT licensed C# compiler, call it C* and make that transition in a language that is definitely no longer C#. I just don't think this is a good example.
For good examples, see the language evolution that has actually taken place which has been community driven in many cases. Records, tuples, the evolution of type constraints have all been started from community feedback, and are backwards compatible.
As a final note, a comment on "aligning with Microsoft's interests". They are interested in two things for .NET: selling Azure hosting, and incentivizing Windows programming. Making the software development experience pleasant, robust, predictable and backwards compatible are the objectives, so devs are incentivized to pay them to program on their OS, target your program to their OS, or run your software on their cloud infra. I really don't see the problem.
I find such hate quite understandable. I might not share it, but I would not put any trust in MICROS~1 either, especially not in a professional setting.
When I develop applications for MICROS~1 platforms, I do it in Java, betting that the Eclipse foundation and a plethora of corporations, including MICROS~1, aren't going to suddenly team up and do something nasty to the language and important libraries. With MICROS~1 as the lone manager of a language and most of its 'ecosystem' this seems like a real risk.
Why? All of the code is MIT licensed. If MS does anything untoward, forking it is trivial.
Look, there are obvious differences in how Java and the JVM are maintained and evolved as compared to .NET, but to continue claiming that these differences really matter all that much is just FUD, to put it politely.
No, it's not. Is Mono independent today?
The only reason I’d use or contribute to Oracle I.P. is if I was paid to do so. The other is porting off Java but that’s legally risky if minima changes. CPAChecker was the one I was thinking of. Past that, I’d try to avoid Java.
Sure, you can compile and host your .NET Core web app on Linux these days, but desktop applications are a completely different beast. You're often lucky if one exists at all, and if it does there's a decent chance it's not designed for "modern" C#-on-Linux. It really isn't a viable option yet, maybe in a decade or two.
https://github.com/dotnet/wpf/issues/2554
That said, there are cross platform, open source .NET UI frameworks out there, including one that is inspired by WPF:
Any evidence for this? From the benchmark game it seems like c# is more performant than go, and measurements are done on Linux.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Which? Just curious.
Some-of the spectral-norm programs seem quite similar:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Sometimes classes aren’t the abstraction I want to use.
That’s why I like kotlin over Java or c#. It’s be nice to have a language like that which targets .net
Also, JVM ecosystem does not offer the ability to provide zero-cost abstractions, which C# does with monomorphized generics. This is a hard requirement for productive systems programming if you don't want the C level of verbosity.
This is true, wasn’t thinking about that.
C# always requires me to use and organize my code into classes and my data into objects.
For example, If I wanted to use higher order functions, I need to wrap functions in callables (Objects).
If I wanted to throw together a quick script to test, I need to set up a program which uses some magic configurations so that I don’t need that entry point class.
Granted I only worked with C# professionally for a couple of years before getting back to a Go shop, but it always felt like trying to avoid thinking in objects was fighting against the language.
Personally, I like just having first class functions which take simple data structures as arguments. Modeling the world as objects just isn’t as clear to me (the old OOP vs FP discussions)
Languages like C# and Java turn me off for that reason.
I used C# for a long time, and when I started using JS, I marveled at the directness of its expressiveness, its lack of ceremony.
What's actually needed is a language as direct as JS, with the bad parts stripped out, types and a proper standard lib added, and compiled to byte-code for performance.
They can call it a different name if they want.
It’s important for how I like to code.
It has always had a fair share of FP features and only gained more as it kept evolving. It had higher order functions partially in the form of method group to delegates conversions since C# 2.0, released 19 years ago and to a full extent in the form of lambda expressions since C# 3.0, which also included LINQ, released 17 years ago. It is a mixed-paradigm language.
On the off chance that this is trolling, I must point out that Java and C# are sufficiently distinct, with the latter leaning more heavily on offering both lower and higher level features like extensive pattern matching, LINQ, structs and pointers/byrefs.
If you do have C# snippets you are unhappy with - please post them. There is likely a better way.
(I noticed you posted about Callables twice, and had a quick search - it appears to be a Godot-specific abstraction imposed on C# due to inadequacy of GDScript it has to interoperate with, and has nothing to do with C# itself)
In one way, they help the adoption in the games industry, in the other hand they introduce so many anti-patterns with their reflection based SDKs, and use of magical methods instead of proper .NET code.
Regardless of the actual word, there is an extra thing I need to do to pass a function to a higher order function as an argument.
I was using this as an example to demonstrate how C#’s support for the language features I like require some ceremony which reduce expressiveness and require me to think about the language more.
Post an actual example (code).
https://learn.microsoft.com/en-us/dotnet/csharp/programming-...
I need to create a delegate for any function I want to pass around.
var func1 = (int i) => i + i;
var func2 = () => func1(1) * 2;
Console.WriteLine(func1(4) + func2());I would also place D and Swift into that bucket, only D never managed to really take off when it had the opportunity, and Swift is clearly at home in iDevices.
They have support for these features, but they require some ceremony to use (like wrapping function arguments in a Callable), which increases friction, signal to noise ratio, and decreases the expressiveness that I want.
I just want to write what I want without having to think too much about how the language wants me to write it. Go is good for that. JS too, but has a lot of historical cruft.
var foo = [1,2,3];
Add on switch expressions, primary ctors, first class functions, no ceremony in Program.cs anymore... it's hard to get more terse and expressive unless you want lots of chained single symbol operators (look at F#).
void myFunc(List<int>){}
myFunc([1,2,3]);
The expression sytanx can infer the type from the caller syntax so you can just inline the array declaration.
The downside of all this is that the only people that know how awesome C# is are people who are already doing C#. Growing the pot and trying to convince people to give it a go in its modern (post dotnet 5/"core") iteration is like pulling teeth. Everyone assumes nothing has changed since like C# 2, and given the fact it's a "boring" language that doesn't have Hacker Hype behind it, people just ignore it.
Every week you have people on HN wishing for something that does exactly what C# does but don't want to give it a shot or admit that C# is _actually_ an incredible language and toolset that does exactly what they want (and more).
https://learn.microsoft.com/en-us/dotnet/api/system.reflecti...
https://github.com/dotnet/runtime/blob/main/docs/design/feat...
https://news.microsoft.com/2001/10/22/massive-industry-and-d...
Anyone with the pre-release MSDN .NET SDK that was initially made available to MSFT Partners, with alpha documentation written in red, has a folder with tons of functional languages that could take advantage of tail calls.
F# only came later into the picture, even considering its OCaml for .NET origin.
As an aside, and as someone who's currently going through materials such as Crafting Interpreters (and who knows what else based on suggestions from r/compilers), is there a, I guess, guide for people who want to implement a compiler that targets the CLR/CLI?
I have a copy of CLR via C# 4th edition, but other than that not sure what else I can reference targeting anything newer than .NET Framework 4
This might be a bit of a learning curve if you don't have experience with writing compilers like that though.
There are other materials that might prove useful should you go down this path:
- A series of blog posts of a student who is writing a Rust to .NET compiler which you could follow along: https://fractalfir.github.io/generated_html/home.html The project itself: https://github.com/FractalFir/rustc_codegen_clr
- A video series on writing a compiler for .NET from scratch: https://www.youtube.com/watch?v=wgHIkdUQbp0&list=PLRAdsfhKI4... Notes: https://github.com/terrajobst/minsk/tree/master/docs
As for CLR via C#, while it has certain facts that remained the same, the ecosystem has changed a lot since then, you really want to target the latest LTS to get best experience and performance. If you liked it, then I can strongly suggest https://github.com/dotnet/runtime/tree/main/docs/design/core... which covers .NET runtime internals.
p.s.: if it's any consolation, the promise in CLR via C# of JIT having the advantage of dynamically profiling code execution and compiling to profile-guided version, tuned to the exact hardware and environment is finally fulfilled, many years later :)
I was recently musing about this on Mastodon[0], commenting on a tweet that said something to the effect of "C++ isn't complex, it just doesn't lie about the complexity of reality". I think Rust is similar, and really any system language has too be. However, there's a space between lying about reality and fully reflecting reality where useful abstractions, such as Rust's lifetimes live. If you hardline the "lying about reality" framing, any abstractions on top of machine code must be ruled out. So while I see where you are coming from, I think lots of space to improve system programming, as Rust did.
> What's really lacking are safe, easy, strong typed general purpose languages that can leverage AOT compilation for high performance and static analysis etc. A language with the learning curve of Python, near C performance and strong safety guarantees of Rust. There is nothing that suggests to me such a language would not be possible. Swift and C# come closest but they are warped by their respective corporate overlords.
I agree about Swift fitting this mold, it's unfortunate that it has failed to gain traction outside of Apple's ecosystem.
To surprise of many outside of .NET community, it has much better performance than Swift[0][1][2] and lets you match performance of C and C++ in areas that matter.
[0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
[2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/... (better performance for short-running benchmarks sensitive to JIT)
[bonus] C# AOT vs Go https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
(no direct comparison but Tl;Dr: ARC and Swift defaulting to virtual dispatch that can't always be elided with WMO results in Swift having much lower (even if more consistent) performance)
Also wow, I stand corrected, Swift is a much stronger contender in Benchmarks Game scenarios than earlier microbenchmarks and research projects that I've seen would suggest. Particularly when using its own SIMD abstraction, it is likely LLVM makes very competent transformations, possibly merging narrower operations into a wider ones, replacing one set of intrinsics with another, etc.
However, still seems to suffer quite a bit, just like Go, in Binary Trees (it's not like C# does the best there either, the winners use bump allocators, and then there's also a 2.6s magic entry by Java, write barrier elision?).
I'd wonder whether that's just me being dim, but if so then I think Bjarne Stroustrup and Herb Sutter are dim in the same way. (Stroustrup, quoted approvingly by Sutter: "Inside C++, there is a much smaller and cleaner language struggling to get out." and "Say 10% of the size of C++ in definition and similar in front-end compiler size. ... most of the simplification would come from generalization.") And it doesn't seem likely to me that they are ignorantly overpessimistic about the complexity of C++.
C++ also does "lie" about reality in a lot of ways, in the sense that the actual compiled program behaves quite a bit differently than a plain reading of the code implies to a nonexpert. The most obvious form of this is the rules for effect ordering. Even the recurring complaints about the compiler doing unexpected things based on the as-if rule and undefined behavior seem to count here.
"C++ Initialization Story: A Guide Through All Initialization Options and Related C++ Areas"
https://www.amazon.com/Initialization-Story-Through-Options-...
"Undefined behavior" is not reality; it's a strange shadow world where anything may happen. The compiler reserves the right to reject your reality and substitute its own - which it will not show you within the language, nor let you choose or hint, nor warn you.
People like to use C and C++ as if they were a species of typesafe macro assembler that always produces predictable machine code, and that breaks down often enough that the world is full of systems-level CVEs.
Most real world security issues have nothing to do with code. Social engineering breaks more security than memory issues. We are also skewed by many of the fundamental programs are written in C and so there is a lot of value in finding holes.
The first thing that crossed my mind reading this comment was Nim. It’s pretty Pythonesque (but that’s kinda what I hate about it.)
Also channels and WaitGroups are easy to misuse (well, that's why there's also an ErrGroup now).
I'd make Golang my 100% go-to language if it got rid of several footguns.
Though let's be real, that's absolutely certain to never happen, they take their backwards compatibility extremely seriously. So we'll have to keep relying on checkers and linters that alleviate some of the pains somewhat.
In fact, most projects I've worked on won't even merge a PR with ignored errors.
Also, `go test -race` is a Go feature, there's nothing wrong with using it.
But it's worth remembering that it's a dynamic analysis that only covers access patterns created by your tests.
The question is: is it harder than it needs to be due to accidental complexity[1] in programming language design. TFA argues that there is still some accidental complexity in the design of Rust that leaves room for an "easier" language that still allows the level of control needed for a systems language.
[1] https://medium.com/@sharkaroo/navigating-complexity-in-softw...
And while Android ART technically isn't Java (the plaform), its mixed execution mode with hand written Assembly interpreter, JIT, AOT with PGO shared via play store is quite cool.
Sounds like Go or Zig, though Go is slightly less safe and slower - in turn being much easier.
Though I'd reach for Go first if I'm working with anything concurrent. Concurrency is hard, and Go really helps with getting it right.
USB Armory is still being sold.
- Strip out the bad parts.
- Add types, more functional features, a proper standard lib, and a standardized module system.
- Allow compilation directly to performant machine or byte code.
They can call it a different name if they want.
- It has simple, straightforward, and pleasant syntax/semantics. Except for a few tweaks here and there, there is no need to re-invent the wheel here.
- It is pretty functional, all that's needed is to add everything-is-an-expression
- It supports multiple paradigms (functional, imperative, OOP)
- Since it has no types currently, a _modern_ type system can be feasibly added, supporting concepts like result/error instead of exceptions, maybe/option, pattern matching, etc.
- It has easy-to-reason about async features
- It has a huge ecosystem; it shouldn't be hard to port many high-quality packages to a similar language
Now, the lower-level features are another matter entirely, but it shouldn't be too difficult to retrofit what we want, like multi-threading (maybe building on top of workers), utf-8 strings, and so on.
...Spoken like a person who's never coded anything meaningful or serious, if you can't even gauge the complexity and man-hours needed to create (much less port) high-quality packages between programming languages.
Talking about "man-hours" tells me you think of software engineering like an assembly line, which it is not.
There are whole lot of factors that might influence the time needed for such porting; the keyword in my comment is "similar". If the new language is a superset of the original language, minus a few warts, I'm not sure what should take so much time. For one thing, the architectural design of the packages (which probably took most of the original effort) remains the same.
The funny thing is that MACLISP could do this back in the early 70s. While Lisp is a dynamically typed, GCed language, MACLISP let you annotate variables to promise that they would only be called on certain primitive types, thus supporting compile-time optimizations for numeric code and not necessarily allocate memory, follow a pointer, unbox data etc. This was critical to make packages like MACSYMA run on machines with performance in the KIPS range.
Although languages like Python support similar declarations or annotations, they aren't used in the same way -- if you want performance you write the module in C++ and give it a python interface.
And woe betide you if you happened to violate the preconditions of one of those annotations. That kind of programming is no better than C with "Segmentation fault. Core dumped."
The problem is that "system programming" or "embedded programming" isn't one thing.
There are people who want to throw around a zillion polygons at 144FPS. There are people who want max performance on x86-64 shoveling around millions of network connections. There are people who need guaranteed real time response or someone dies. There are people who need to stay asleep all the time to conserve battery and only wake up once every 5 minutes to chirp some RF.
These people do not have the same needs in a language yet they almost all wind up shoveled into C.
Why is that tho? There are better languages for each of those applications. What do you think, is it business requirements? Hardware? Regulation? Legacy code?
C is a great choice because I can hire people who know C and get good results. A domain specific language will not be much better than C for any of the above. It turns out that all the hard problems listed above are similar enough that someone good in one will be good in them all and so I can hire C experts from any field and get going fast without having to teach them my domain specific language that isn't really much better.
I do not know the best way for any modern language to take to any other modern language - but 1972 C is a big negative to them all.
On other systems we have XPC, COM, AIDL, TIMI, MSIL.
That said I fully accept your correction. (even though it doesn't apply in my world)
Umm, this is MACLISP we're talking about -- you know the types at runtime and can pass the unspecialized argument to an ordinary function. A kind of generic function I suppose, though we didn't use that vocabulary back then.
Just because this was 50 years ago doesn't mean that programmers were morons.
Consequently, whatever you sent in was assumed to be what the annotation was. There was no "compilation phase" to ensure that the type at the call point actually matched the type in the annotation. If those differed, then your program would just go boom.
Now, it could be that I'm remembering something other than MACLISP. It's been a lot of years.
I tried to be clear by using the modern terminology “generic function”.
If you compile a function with those annotations you’d end up with two functions: one that could be called if you knew the types at compile time and one that did not.