What's next for language design? (2017)
graydon2.dreamwidth.org
graydon2.dreamwidth.org
Computers ought to be our helpers which could understand our plans and implement them properly. Instead we are burying ourselves and those who will come later in what is essentially a deep math. And our software doesn’t even get faster or better, it just fits whatever performance:annoyance ratio is acceptable at the end of the day.
I think, and this is more of an intuitive rather than an informed guess, that the current practical languages are still too low level and that may be one of big reasons why software development is so complex. You probably feel it too, when you have to either A) go find, copy and fine-tune yet another boilerplate or a snippet, making it again too hairy to use as is next time, or B) use a standard solution which almost implemented another programming language on top of its configuration and isn’t really much easier to use than if it were just a from-scratch code. Shovels get better, but we are still digging careers with them.
And we had predecessors of a theoretical language which could be high-level enough to make programming great again. E.g. FoxPro with all its DOS restrictions was a platform where one could read, write, iterate, update data, never thinking how to map classes to tables or forms, or how to make it run. Or Delphi, in which most students could write client-server apps by dragging tables, queries, databases and forms right from a toolbar. The entry complexity skyrocketed since then or, more precisely, returned to “serious” levels of heavy C++ setups which only seasoned programmers by trade could manage without pulling hairs.
I don’t think that human[ity] will become any significantly smarter in the near future, and that, together with our all time high software demands, means that we will experience a disabling shortage of software developers until we fix our internalized tolerance to the increasing complexity.
The points on modules and effect systems are totally in line with creating a shallower learning curve.
My reading of his point on formal specifications was that it was important to design software that works as it behaves (like compilers that compile the languages they say they do, paraphrasing). That's not a burden on users of the language directly, more on the creators.
It only prevents some misuse classes of APIs.
I think what you want more dearly is to express atomiticity of functions and (automatic) rollback on error to a "known good state".
Unrelated to typing, algebraic effect handlers probably have the most ergonomic implementation of async I have ever seen (looking at the OCaml multicore previews).
Yes. That is compiler provided functionality on top, which looks excellent.
Just dropping in to say strong agree.
The biggest stumbling block for me has been that the effect handlers are captured as part of the continuation; I was expecting them to need to be manually re-established every time the continuation is resumed. The way it works makes sense when you always `continue` within the body of an effect handler, but it feel a little unfortunate that there's no way to change the handlers for future resumptions.
Other than that, the possibilities for userland coroutines, dynamically-scoped variables, and centralization of state (like `ref` but provided by a scoped allocator) are incredibly exciting. I've wanted a system like this for a long time, and for some use-cases (scheduling of concurrent simulation tasks) I had to abuse threads and thread-locals to get the separation I wanted.
This is because many insist on having one giant programming language to do everything both low-level and high-level together. Rust just follows C++ along this same path. The resultant language will always be a complex compromise that is suboptimal for almost everything. For example, both Rust and C++ have too many high-level abstractions to easily reason about where allocations will happen and their manual memory management makes lambdas and async painful. I personally would favour the use of multiple languages. Such an approach is commonly emerging anyway due to practicalities, e.g. Python/C/Fortran use in machine learning.
Whenever you have a programming language with multiple goals (or, let's say, more than two distinct goals) - it's almost certainly going to be suboptimal for any individual one of them.
The questions are:
* Whether choice of goals is worthy/reasonable; and
* Whether a good balance and compromise has been struck by that language's specifiers and implementers.
Also remember that languages evolve; and if one of the language goals is backwards compatibility (which is the case for C++, less so for Rust) - then part of the compromise is that they have to support multiple older and less-recommended idioms and facilities while offering newer, hopefully better ones.
This is very far from true. Like, not even the tiniest bit true.
It is easy to program in C++ or Rust and pay no attention at all to where allocations happen. Almost all application-level coding in those languages is written exactly that way. Even in programs where such attention is needed, the overwhelming bulk of the program, such as all the initialization and setup code, gets away with completely ignoring allocation. Certain libraries whose value proposition includes having been heavily optimized need to pay attention to memory placement, but probably more than half of libraries would only ever be used in places where performance doesn't matter at all, and the author knew it.
Anyplace where you do need to pay attention to details of allocation, or of atomic event ordering, or of interrupt context, you are not obliged to use every last bit of abstraction your language enables. It is totally allowed to drop to a C level of specification. (In Rust you might even put some of it in "unsafe" blocks.)
Rust and C++ do impose inconveniences for various reasons -- C++ mainly because of backward compatibility, Rust mainly for extra static enforcement -- but the inconveniences are completely unrelated to the occasional need to pay attention to memory placement.
Where existing languages do still demand low-level attention is in obliging the programmer to be aware of control flow, and what thread is executing which bit of code. They are only starting to loosen that. It used to be that you didn't care because there was only one thread. Then, we made threads, but not too many, because of numerous costs. Now, we would mostly like our programs to use as many threads, scheduled any which way, as would be useful.
>> do everything both low-level and high-level together
> This is very far from true. Like, not even the tiniest bit true
Does not really align with:
> It is easy to program in C++ or Rust and pay no attention at all to where allocations happen
(This is what high-level languages offer)
> It is totally allowed to drop to a C level of specification.
(This is what I mean by low-level)
I would also argue that in Rust the memory management unfortunately does affect the ergonomics of lambda and async, for example the need to explicitly pin memory.
I really don't mean to criticize all the recent effort that has gone into Rust, it is certainly a huge improvement over C++. But it's far from perfect and I'm arguing that is probably not the fault of any of the individual choices, but rather the language goals. I would like to see a future where we use more specialised languages; rather than the many languages that can both peek/poke bytes and implement the full lambda calculus. More specialised, simpler, languages are much more amenable to static analysis and formal methods.
I might be biased, but I'm excited for Zig and Cone, which I think will make it easier for newcomers to the low-level space.
The top of the learning curve is the point where the language no longer has any help to offer, and that should be as high as feasible.
The difficulty with Rust is that it's "batteries not included", which is a term we use in software for the equivalent of a car sold with no wheels, body panels or seats. Sure there's a package manager, there's also an auto parts catalog.
Can’t disagree, but where are the languages for little software? For example, I write lots of small (100-1000 LoC) command-line tools. I’m not a full-time programmer, so I don’t have time to learn a complex language - what I ideally want is executable pseudocode.
Are there languages that fit that description? Python used to, but even it has now become very complex - because its designers are all employees of big companies working on big software.
A carefully balanced quantity of DWIM (Do What I Mean) without too much WAT, while having awesome error messages to help you get going and continue to be productive.
We learn our lessons slowly. The winning language always surfaces itself in the form of useful software. If it’s true that these new language constructs are useful, pain-free, then we will surely see the ubiquity of those languages and the corresponding software with it.
If it’s not useful, then nothing useful will come of it. It will be just another exercise in technical virtuosity, a master violinist playing a technically profound tune that no one likes to listen to. You’ll just have to enjoy it for the feat that it is, but it’s not something people will want to sit through regularly.
Cheers, design away, I’ll see you on the other side of the cacophony.
Have you seen https://enso.org? Personally, I'm strongly hoping for it to become a modern contender in such space (of making programming easier and faster, for non- and programmers alike)
Anecdotal side note: I recently visited a archaeological museum on greek civilization. Some of the artifacts were 6000 years old and I was really surprised how ... delicate and "modern" for the lack of a better word ... the earthenware looked.
That really made me think that humans back then were just as intelligent as we are today. I think that may be a fact that may be supported by biological evidence? Brain size comparisons of skulls maybe?
Either way, if humans have not been less intelligent back then, there is only technical progress left.
But, it will also bring more people into the field that will likely increase the amount of poorly written code that needs to be maintained.
What happened to all the talk around making the compiler do all of the work?
I guess, instead of seeing it as an increase in complexity, I see it as a tradeoff of increased learning curve for decreased complexity.
You’re spot on about the labor market side though. More learning curve for the best tools has tons of labor market implications.
I’d bet the end game is more computing being in the general education tracks to deal with the overhead, as well as a wider (hopefully continuous) spectrum of computing tools for people ranging from those using computing as a small part of their work to those using computing for all of it. Like gradual typing. You don’t need to add types to python, but you can add mypy. Gradual depth, maybe.
Every C or Java programmer writes a function definition in terms of a tuple of parameters, despite not having first class tuples. Functional programming languages can also return tuples, but where are the anonymous records? These would allow defining functions with named arguments essentially for free.
So while there may be some use of structural subtyping, I don't think it's codebase-wide.
t -> bool
How would you assign a nominal type to the result of a SQL query?
So there are arguments on both sides. My point was that structural types can always easily be turned into a nominal type, but not the other way around. Therefore they should be the default.
For example, although Java method definitions are essentially defined in terms of tuples, function classes and objects in Java are all entirely nominal.
com.google.common.base.Predicate<String> guavaPredicate = ...
java.util.function.Predicate<String> predicate = guavaPredicate::apply; let foo {x, y} = x + y
in foo {x=1, y=2}
{ x:int, y:int } -> int
Recent versions of Java (Java 8+) avoid structural types but provide structural conversions + inference.
We will see with the recent introduction of named records/tuples goes in the same direction or not.
Of course, backward compatibility means those are optional, but the designer of the template decides. Existing libraries are all structural, but new ones will rely more on nominal meta-typing, and error messages for those are clean.
There will be pressure to retrofit older libraries, so that error messages from misuses of those get nicer too. Gripes from users who had successfully abused those libraries will be heard.
Nominal typing of generic code would be what most other languages that supports generic programming use. For example Rust has traits where one defines a named trait and must explicitly specify that a type satisfies a trait by name.
In C++ there is no such means, a concept does strictly structural checks (via constraints) to determine if a template satisfies a concept, no names are involved. In other words, a type T satisfies a concept C defined by a constraint R if and only if a T can be substituted for C's placeholder type during the evaluation of R.
For an example see std::tuple_size/tuple_element and their relation with (ironically) structured binding.
In Rust, a generic is only right or wrong, in isolation. In C++, how it behaves, including whether it compiles, depends on how you choose to use it. That makes C++ templates strictly more powerful than Rust facilities. Rust could relax some restrictions, selectively, to gain that power, and might someday.
Preventing the "compiler vomit" is a main selling point of Concepts, and it works. It is still possible to subvert it, if you work at that (then you are no worse off than before). So, don't.
When one works with teams our options don't work in isolation.
As for concepts working in preventing compiler vomit, there is still plenty of work to do in 2021.
There is always more work to do. That is not a failing, it's just life.
That is what defines modern C++, a standard full of political compromises, slowly becoming a niche language.
You wish C++ were becoming a niche language, but in fact its usage is still growing by leaps and bounds, as it has been continually since C++11 came out. Every week more people pick up C++ than the whole population now employed coding that other language. That will be true next year, too.
The fact is that it has become a niche language for GPGPU programing, OS drivers and embedded standards like AUTOSAR, everywhere else another language takes the crown jewels, iOS, Android, macOS, ChromeOS, Windows Apps, Web, cloud computing infrastructure,... where is your growing C++?
So much that Apple and Google are now rather focusing on their own languages, with C++17 being good enough for their own purposes, hence why clang is trailing its ISO C++20 compliance.
When everyone brings their own agenda to the table regarding implementation details it is politics, not only concepts, contracts, ABI breaks (god forbid!), reflection, networking, graphics,...., definitly politics.
But hey, it is growing on GPGPU, HPC, Machine Learning libraries for Python, LLVM/GCC implementation language, so there is that.
To be fair, C++ templates are not structural types, they are (together with C++ function overload resolution) an (almost) purely functional untyped programming language, producing C++ programs as output.
OCaml also has polymorphic variants. Which could also be called anonymous sum types.
If you assigned something to the system clock, it would act like an interval timer, for example.
The language can help with debugging by providing traces of execution that you can record and replay to inspect. Then you can query the trace to answer any questions you have about the execution of the program.
It gets messy fast, and I absolutely would not want to debug a big program that was full of implicit dependencies like this. This was tried in Java FX. AFAIK it was a total disaster.
The issue with reactive, declarative languages is they have not been given the same opportunity to optimize. We've devoted a lot of resources to making imperative programming more accommodating despite all of its warts, but with declarative programming so far we have decline to go through the same process.
Declarative reactive programming has proven to have a lot of potential. After all, if we call Excel a programming language (which it is), then reactive programmers far outnumber those of any other style. We should take that direction as far as it can go. Mine it for every good idea the same way imperative programming has been mined. Sure maybe JavaFX was a total disaster. But why can't its successor be better? Why exactly was it a disaster? What could potentially be done to improve that?
a = 10
b = 20
c = a * b
s = "{c}"
widget.display(s) -- will show '200' at the start
a = 2 -- c = 40 and widget shows '40'
a::b*c; b:2; c:3; a results in a being equal to 6, and if you update b or c a will update too.
* Decoupled allocation, to use any allocator with any existing code. Cone has some really cool things on the way here, where we can package up allocators into modules and others can import them and use them [0]. Odin is also trailblazing here, with its implicit allocator "context" parameter [1]. Zig does this in an explicit way in its standard library [2].
* First-class isolated regions, where certain areas of memory cannot point to each other, but within a particular region we can freely mutably alias. Vale [3] and Verona [4] are doing some interesting experiments here.
* Better static analysis, to close the gap between memory-managed languages and C. Lobster's algorithm is very impressive here [5], and Vale is attempting an "automatic Rust" approach [6].
* True "structured parallelism", where we can "freeze" all existing memory and freely access it in parallel from a loop's body. Pure functional languages might be able to do this, but it would be interesting in an imperative language. Rust can almost do this, but can only share things with Sync.
* Blending the borrow checker with other paradigms. Cone is going all-in on this concept [7], and looks really promising. The pieces are also there in D and Vale, but we'll see!
I also have some things on my wish list (full deterministic replayability!) but they aren't really on the horizon yet.
(Disclaimer: I work on Vale, which aims to bring many of these features into the mainstream)
[0]: https://cone.jondgoodwin.com/coneref/refregionglo.html
[1]: https://odin-lang.org/docs/overview/#allocators
[2]: https://ziglearn.org/chapter-2/
[3]: https://vale.dev/blog/zero-cost-refs-regions
[4]: https://www.microsoft.com/en-us/research/project/project-ver...
[5]: https://aardappel.github.io/lobster/memory_management.html
Edit: See https://docs.rust-embedded.org/book/collections/index.html#u... for a through explanation
The biggest problem is that there's some awkwardness around RAII; I'm not sure whether that could have been avoided with a different approach.
Of course, ideally you'd want it to be compatible with the standard-library APIs that allocate. This is implemented, but is not yet at the point where they're sure they won't want to make backwards-incompatible changes to it, so you can only use it on nightly. https://doc.rust-lang.org/stable/std/alloc/trait.Allocator.h...
Or are you suggesting that the choice of allocator should be dynamically scoped, so that allocations that occur while the bump allocator is alive automatically use it even if they're in code that doesn't know about it? I think it's not possible for that to be memory-safe; all allocations using the bump allocator need to know about its lifetime, so that they can be sure not to outlive it, which would cause use-after-free bugs. I'm assuming that Odin just makes the programmer responsible for this, and if they get it wrong then memory corruption might occur; for a memory-safe language like Rust, that's not acceptable.
I've never used Odin, so I don't know whether/how they'd keep it safe.
[0]: https://docs.google.com/document/d/1243br9VVluZN0ZD9MVKSQMAw...
On the other hand, if you required an API along the lines of bumpalo::boxed, and also were willing to add a bit more runtime overhead on top of that (for the tracking bits and the allocation count), then this could be done in Rust as a third-party library. Each executable that transitively depended on it would have to opt in with #[global_allocator], though. Also, I personally would rather have an API where the compiler makes sure I don't screw this up, than one where the runtime crashes my program if I do; this is generally a common sentiment in Rust, and points towards a bumpalo-style API.
Edit: Spelling. PS: Though might also be an awesome way to shoot yourself in the foot.
And I think your post deserve to be at the top. I wish I could super upvote.
I've mostly been in the https://reddit.com/r/ProgrammingLanguages subreddit and its discord server, and chat with these folks a lot, about interesting innovations.
It's a fascinating time to be into programming languages!
(I'm especially interested in new memory management techniques)
The devil on my shoulder is kind of telling me that it's just another example of FP people slowly re-inventing a form of Object Orientation, though. But I think that's a good thing. A rose by any other name...
Yes, I know that for us "real" programmers this is dreadful, but it would make programming more accessible and this could have interesting network effects on our society.
I'd imagine that this language would have some traces of visual programming (like Enso[0]) and the denseness of APL[1].
[0] https://enso.org [1] https://aplwiki.com/wiki/Simple_examples
So, first, mobile devices need to be busted loose from the gatekeepers.
Rather, I think that PL design headroom is largely in new programming domains, including:
Distributed systems. Right now, these are cobbled together between single processes written in conventional programming languages; RPC & serialization frameworks; container or cloud orchestration mini-languages; and a lot of proprietary tooling. There's no unified way of expressing "this computation occurs on this box", while abstracting away the details of communication & provisioning and still providing robust handling for debugging and the complex failure modes that distributed systems introduce.
Machine-learning. This is handled through frameworks (TensorFlow, PyTorch, JAX) that basically provide EDSLs in a host language, usually Python. But a lot of them would really benefit from language-level support; for example, differentiability is an important concept for SGD, and yet is bolted on with a bunch of kludges in these frameworks. There's also the whole issue of how to store & version training data; how to collect it; how to easily support feature extraction and labeling as part of a developer's workflow; how to run in development vs. production mode; and so on.
GPGPU. Your options here are basically CUDA, Futhark, and a few other niche languages. Yet the underlying hardware and computing model is fundamentally different from single-process CPU code, or even multithreaded concurrent CPU code. It seems like there's a lot of headroom by thinking in terms of the operations a GPU provides and then exploring the space of problems that could be solved with that much computing capacity.
Smart contracts. Solidity paves the way here, but Solidity is still pretty kludgey, and makes it easy to shoot your foot off in a far more expensive way than C++. A single vulnerability can cost literally billions of dollars, and often in a very public (and permanent) manner. They're also (by default) immutable, so you can't fix bugs after the fact, and introducing mutability to them also often introduces gaping security holes. You have to worry about gas costs, and each computation operation is a lot more expensive than in traditional computer programs.
Basically, headroom for improvement in programming languages comes from handling things that are not handled in languages right now, where solutions get cobbled together from a bunch of libraries or custom code. That's how it's always been with product design: the products that most need to exist are those that cover problems where no existing product exists, not ones that are minor improvements on existing products.
I suspect that progress on this front will overlap a bit with the previous category, as more often than not you run machine learning workloads on GPU anyway.
I think the note about C++ was specifically about member functions qualification, where currently it isn't possible to parametrize over const this vs non-const this. The "Deducing This" [1] proposal will hopefully fix this [2] and other issues.
[1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p084... [2] pun fully intended.
Variable names are too important for the computer to be totally oblivious to their meaning, yet this is how all programming languages work.
> It's "small" but rather important not to mix units!
With unit of measure, it's possible to easily/simply create types like CustomerId, which is actually a Guid/Int under the covers. Means you can't pass a CustomerId to something that expects an OrderId. These types are erased at compile time, so there's zero effect on runtime performance. It's lovely.
> ... does the system of function types and effects scale to expressing polymorphism over the mutable-ness of a type ...
That's a very thought-provoking handful of sentences. Anyone know of work on this question/area?
Because, a>b could also mean a should become 6. So you need to understand the intent to decide what you should do. In that sense, it's not even prolog, it's "worse" (read: harder to achieve).
What you propose is essentially what some languages like Idris already offer: general purpose programming with constraints that allow the compiler to generate an implementation, given that the constraints are precise enough.
Hah, I've seen that and had no name for it. It's definitely a thing, and an annoying one, when designing a language in committee.
Alan Kay compared out current practises as akin to pyramid building; very little is understood about structural engineering properties of the materials, we just place block on top of block. He argues that there is no programming equivalent of the arch.
But suppose that big blocks of stone is all there is. I recall Linus Torvalds talking about one of his proudest achievements. I don't remember specifics, but it was some memory allocator IIRC. All of this was achieved at the microscopic level using C. It's one human sitting down and figuring out exactly what was required to solve the problem, which bits to twiddle, and which bits to leave alone. No shortcuts.
I would argue that programming is not like other engineering disciplines. It boils down to this: we can create specific solutions to specific problems, but we can't create specific solutions to general problems. That is to say, we can create programs like spreadsheets, do accounting, model airflow through turbines, whatever you want, because we understand the problem domain, and it is constrained. As to "how do I write my program", well, what program are you trying to write?
Compilers cannot reason about the structural integrity of a program. They are general purpose. They do not have any domain knowledge about the type of program you are trying to construct. They can therefore only tell you if a program is syntactically valid, not whether the program is flawed at the domain level.
Can we give compilers (or some other near equivalent) domain knowledge? In an exceptionally limited sense, perhaps. I have in mind configuration tools that some microcontroller vendors release. These programs have domain-specific knowledge about how their microcontrollers can, and cannot, be configured and for what purpose.
They are extraordinarily constrained in scope. The hard work of actually getting the hardware to do what you want it to do is left up to you.
So we shouldn't expect too much out of any domain-specific solution. Their scope is highly limited. We shouldn't expect many to be available.
So I'm not expecting much in the way of paradigm shifts in the near future.
The author appears also not to know formally proved and domain specific languages, which are "Effect systems" (they check if conditions are uphold) to generate C code. Generally life is a tradeoff and the same holds for showing semantically expressive properties (or one needs a proof system).
Most languages rely on LLVM and its linker with linker speak, which are both not formally proven to be consistent and both have no model. The spec also does only describe basic hardware abstraction and without a formal model from CPU vendors this wont improve much.
"Cost-model type systems": only sel4. check the blog of the head with papers why https://microkerneldude.org/
Generating C code is pointless unless a C compiler is the only way you have available to emit machine instructions.