Hacker News Reader

> - A Language To Encourage Modular Monolith

> But what's being proposed here?

I read it, and to me it seems like they're worried about the wrong things. As I understand it, they're worried about the difficulty and hassle of calling unrelated code in the monolith, and proposing things that would make it easier. But that's wrongheaded. Monoliths don't suffer because it's too hard to reuse functionality. They suffer because it's too easy. Programmers create connections and dependencies that shouldn't exist, and the monolith starts to strangle itself (if you have good tests) or shake itself to pieces (if you don't) because of unnecessary coupling. You need mechanisms that enforce modularity, that force programmers to reuse code at designated, designed module interfaces, not mechanisms that make it easier to call arbitrary code elsewhere in the monolith.

In my opinion, a great deal of the success of microservices is due to the physical impossibility of bypassing a network API and calling the code of another service directly. Because of this, programmers respect the importance of designing and evolving APIs in microservices. Essentially, microservices enforce modularity, forcing programmers to carefully design and evolve the API to their code, and this is such a powerful force for good design that it makes microservices appealing even when their architectural aspects aren't helpful.

A language that made it possible to enforce modularity in a monolith as effectively as it is enforced in microservices would make monoliths a no-brainer when you don't need the architectural aspects of microservices.

This is literally the core tenet of OOP, and arguably Java and friends did a decent enough job of that via encapsulation and visibility modifiers.

Not really. Look at any open source Java library, and you'll see that unless it's small enough to implement in a single package, it probably contains public classes that aren't meant to be part of the public API but are declared public so they can be used in other packages in the library codebase.

That's why they introduced a module system for Java in Java 9. It sounded pretty cool to me when they announced it, but it was a bit too late to make much difference in the Java ecosystem (Java 9 came out 21 years after Java 1) and I haven't heard much about it since then.

The module system is used extensively by the core java ecosystem (e.g. the jlink tool will create lean "JREs" based on which modules are actually used), and is getting more and more common in projects that are using a relatively modern Java version.

Besides, it makes sense to encapsulate at different scopes - a class, a package, a module, a library/application. These are "abstractions" on different levels, semantics, constraints.

> Capabilities ... They're not really compatible with C's flat memory model ... Capabilities mean having multiple types of memory

C is not really dependent on a flat memory model - instead, it models memory allocations as separate "objects" (quite reniniscent of "object orientation in hardware" which is yet another name for capabilities), and a pointer to "object" A cannot be offset to point into some distinct "object" B.

> A Truly Relational Language

This is broadly speaking how PROLOG and other logic-programming languages work. The foundational operation in such languages is a knowledge-base query, and "relations" are the unifying concept as opposed to functions with predefined inputs and outputs.

Possibly the nearest to applying capabilities to C is pointer authentication: https://lwn.net/Articles/718888/

(This is one of those times where the C memory model as described in the spec is very different from the mental PDP-11 that C programmers actually use to reason about)

C's file descriptors are capabilities.

CheriBSD is definitely closer: https://www.cheribsd.org/. Or one of the other CHERI projects.

> In general, while I can’t control how people react to this list, should this end up on, say, Hacker News, I’m looking more for replies of the form “that’s interesting and it makes me think of this other interesting idea” and less “that’s stupid and could never work because X, Y, and Z so everyone stop talking about new ideas” or “why hasn’t jerf heard of this other obscure language that tried that 30 years ago”. (Because, again, of course I don’t know everything that has been tried.)

OK, a few things that languages ought to have:

- Everything except C now has standard strings, not just arrays of characters. Almost all languages now have some standard way to do key/value sets. What else ought to be standard?

-- Arrays of more than one dimension would be helpful for numerical work. Most languages descended from C lack this. They only have arrays of arrays. Even Rust lacks it. Proposals run into bikeshedding - some people want rectangular slices out of arrays, which means carrying stride info around.

-- Standard types for 2, 3 and 4-element vectors would help in graphics work. There are too many different implementations of those in most language and too much conversion.

Things to think about:

- Rust's ownership restrictions are harsh. Can we keep the safety and do more?

-- The back-reference problem needs to be solved somehow. Back references can be done with Rc and Weak, but it's clunky.

-- Can what Rust does with Rc, RefCell, and .borrow() be checked at compile time? That allows eliminating the run-time check, and provides assurance that the run-time check won't fail. Something has to look at the entire call tree at compile time, and sometimes it won't be possible to verify this at compile time. But most of the time, it should be.

-- There's a scheme for ownership where there's one owning reference and N using references. The idea is to verify at compile time that the using references cannot outlive the owning one. Then there's no need for reference counds.

-- Can this be extended to the multi-thread case? There have been academic demos of static deadlock detection, but that doesn't seem to have made it into production languages.

-- A common idiom involves things being owned by handles, but also indexed for lookup by various keys. Dropping the handle drops the object and removes it from the indices. Is that a useful general purpose operation? It's one that gets botched rather often.

-- Should compilers have SAT-solver level proof systems built in?

-- Do programs really have to be in monospaced fonts? (Mesa on the Alto used the Bravo word processor as its text editor. Nobody does that any more.)

-- There's async, there are threads, and there are "green threads", such as Go's "goroutines". Where's that going?

-- Can we have programs which run partly in a CPU and partly in a GPU, compiled together with the appropriate consistency checks, so the data structures and calls must match to compile?

-- How about "big objects?" These are separately built program components which have internal state and some protection from their callers. Microsoft OLE did that, some .dll files do that, and Intel used to have rings of protection and call gates to help with that, hardware features nobody used. But languages never directly supported such objects.

So there are a few simple ideas to think about.

> Do programs really have to be in monospaced fonts?

Of course not. I've been using a proportional font for at least 10 years and I'm still in business working on code bases shared with developers using monospaced fonts. Both work, none disturb the other, proportional is easier to read as any book can demonstrate. Alignment doesn't matter much.

How do you deal with writing code with multicursors when you have to type the same thing multiple times? With monospace I just ctrl+alt+down a couple times on aligned text and then type. With proportional fonts I don't suppose it's easy to align text exactly, so do you just not use multicursors or is there a solution you came up with that works?

I’ve been a professional programmer for 17 years and have never used multicursors. I don’t even fathom under what conditions you’d want to. I use Find and Replace.

Everyone programs a little differently. I often use it when e.g. using intrinsics and I want to change types. Find and replace isn't especially helpful when they're different names with substructure you need to modify locally.

Yes and ctrl-V in vim would be super awkward.

I don't use multicursors much, not every year. I'm using emacs. I register a sequence of keys and apply it multiple times with control e, control e, control e etc.

Not sure it counts as "multicursors" but overwriting a rectangle using C-x r t (string-rectangle) is pretty handy in emacs.

> - Rust's ownership restrictions are harsh. Can we keep the safety and do more?

https://www.languagesforsyste.ms/publication/fearless-concur...

> -- Should compilers have SAT-solver level proof systems built in?

They already do. The exhaustivity checker in Rust (and functional languages) is equivalent in power to SAT.

C has arrays with more than one dimension. (but no direct support for strides). Of course, it is just the same thing as arrays of arrays.

Are C multidimensional arrays guaranteed to be contiguous in memory? In practice they are, but can one iterate through them just by incrementing a pointer which points to the first element without UB?

Yes. All arrays in C are contiguous. Multidimensional arrays are just arrays of arrays, and are therefore also contiguous.

Source: https://stackoverflow.com/questions/36647286/are-c-multidime...

Yes, but is one allowed to move a pointer inside it as they see fit? On a one-dimensional array, one can iterate through it starting with a pointer pointing to the first element and ending with a pointer pointing one position past the last element (which the user is not allowed to dereference). For multidimensional arrays, the element type is an array too (with a smaller rank than the original one), so one could perform that type of iteration with a pointer to an array. My question is whether a pointer to the underlying scalar type can freely move inside the multidimensional array without UB, since it may have to actually leave the array it was originally part of. If that's not allowed, how could one build slices and other view types?

Yes, you can do that, it's fine as long as you stay within the bounds of the indexes. Under the hood, it's a single contiguous block of memory.

Although at least with 2d arrays I prefer to just use a 1d array and index it with [x * width + y], because one problem with multidimensional arrays in C is they need multiple allocations/frees.

Why would you need multiple allocations?

edit: Isn't it just:

  float (*arr)[m][n] = malloc(sizeof(*arr));

You have to allocate the size for m and n, because C arrays decay bare pointers.

But it seems to work, which I didn't expect[0]. It also seems like you have to de-reference it to use it which is weird.

[0]https://onlinegdb.com/HwX-WTL5t

Double indirection arrays with multiple allocations are 25 years obsolete (ok, there are some use cases) but since C99 we prefer to do it like the parent.

In your code link you over allocate memory, sizeof *arr is enough and you need to dereference like with (*arr)[i][j]. You need to dereference it because it is a pointer to an array, if you dereference you get an array. You can also let the first dimensions decay then it looks like:

  double (*arr)[m] = malloc(n * sizeof *arr);
  arr[i][j] = ...

but this is not as safe because the bound of the outermost dimension is lost. (Edited)

> Everything except C now has standard strings, not just arrays of characters. Almost all languages now have some standard way to do key/value sets. What else ought to be standard?

I think that character strings should not be restricted to (or generally expected to be) Unicode (although using Unicode and other character sets will still be possible).

I also think that key/value lists should allow any or most types as keys, including references to objects. (PostScript allows any type to be used as keys except strings (they are converted to names if you use them as keys) and nulls.)

I think that big integers (which can have a program-specified limited length in programming languages with typed variables) and arrays of fixed-length records (which C already has; JavaScript has typed arrays which is a more limited implementation of this) are also things that would be helpful to include as standard.

> Arrays of more than one dimension would be helpful for numerical work.

I agree with this too; it is a good idea.

> Standard types for 2, 3 and 4-element vectors would help in graphics work.

This is probably helpful, too, although they can be used for stuff other than graphics work as well.

> Do programs really have to be in monospaced fonts?

No, but that is due to how it is displayed and is not normally a feature of the program itself. Many people including myself do use monospace fonts, but this should not usually be required.

> There's async, there are threads, and there are "green threads", such as Go's "goroutines". Where's that going?

I had read about "green threads" and I think that it is a good idea.

> How about "big objects?" These are separately built program components which have internal state and some protection from their callers. Microsoft OLE did that, some .dll files do that, and Intel used to have rings of protection and call gates to help with that, hardware features nobody used. But languages never directly supported such objects.

I also think it is sensible to have components that can be separated from the callers, and that operating system support (and perhaps hardware support) for such thing might be helpful. I would design a programming language for such an operating system that would directly support such objects.

> - Everything except C now has standard strings, not just arrays of characters.

Zig's strings are null-terminated arrays of bytes. It's an unfortunate mistake from an otherwise interesting language.

Other than comptime, most of Zig ideas were already present in Modula-2, @ all over the place gives a "I miss Objective-C" flavour to it, and only source code ecosystem on a language supposed to be a systems programming language?

> - A Truly Relational Language

> Well, in Python, almost everything is a key/value store.

Why would that be anywhere near an adequate substitute? KV stores are not relational, they don't support relational algebra. KV stores in PLs are common as dirt, so if they were relevant to the question of ending relations in a language I think the author would have noticed.

Commercial image based systems have had source control like management for decades.

If anything many "modern" low code SaaS products are much worse in this regard, than what Lisp and Smalltalk have been offering for years.

> But what's being proposed here?

He proposes that there is a need for a way to connect modules, i.e. dependency injection, without the modules having explicit knowledge of each other, with compile-time verification that the modules being connected are compatible, without the interface song and dance.

Many of these things (not only what you describe here but also the linked article) are stuff that I had intended to be available in the built-in command shell (called "Command, Automation, and Query Language", which is meant to describe some of the intentions) of an operating system design, so that they would have support from the operating system.

About capabilities, I think that capabilities should be a feature of the operating system, although hardware support would be helpful. However, I think that it could be done with tagged memory, without necessarily needing multiple types of memory, and programming languages such as C could still be capable of using them (although some things might not work as it would be expected on other computers, e.g. if you try to copy a reference to a capability into a memory area that is expected to be a number and then try to perform arithmetic on that number, the program is likely to crash even if the result is never dereferenced).

However, my idea also involves "proxy capabilities" too, so that you can effectively make up your own capabilities and other programs receive them without necessarily knowing where they came from (this allows supporting many things, including (but not limited to) many of the idea of "divergent desktop" of Arcan).

> The OP has a good criticism of why this is a bad idea. It's an old idea, mostly from LISP land, where early systems saved the whole LISP environment state. Source control? What's that?

Symbolics Genera can save (incremental and complete) images (-> "Worlds"). The image tracks all the sources loaded into it. The sources/files/docs/... of the software is stored on a central (or local) file server.

I can for example start an initial world and load it with all the wanted software in the various versions I want. Maybe I save a new world from that.

I can also start an pre-loaded world and incrementally update the software: write patches, create new minor/major versions, load patches and updates from the central server, install updates from distributions, ... Maybe save new worlds.

The "System Construction Tool" tracks what code is loaded in what version from where.

> The OP has a good criticism of why this is a bad idea.

They simply assert "twiddling a run-time variable for debugging in your staging environment can propagate straight into a bug on production".

As-if straight into production without re-testing.

> Source control? What's that?

"ENVY/Manager augments this model by providing configuration management and version control facilities. All code is stored in a central database rather than in files associated with a particular image. Developers are continuously connected to this database; therefore changes are immediately visible to all developers."

https://www.google.com/books/edition/Mastering_ENVY_Develope...

~

1992 Product Review: Object Technology’s ENVY Developer

http://archive.esug.org/HistoricalDocuments/TheSmalltalkRepo...

> Capabilities > > Capabilities mean having multiple types of memory. Might come back if partially-shared multiprocessors make a comeback.

I found this description amusing because all modern memory safe languages have capabilities, and they all have multiple types of memory: that's what an object is! Memory safety partitions memory into different types, and object references are capabilities!

What languages do next is where they break the security properties of capabilities: they add "ambient authority" and "rights amplification". Quick primer:

Ambient authority is basically the same problem as globally mutable state. Globally mutable state impedes modular reasoning about code, but if that state also carries authority to do something dangerous in the real world, like launch missiles, then it also impedes modular reasoning about security for the exact same reasons.

Rights amplification is the ability to turn a reference to an object with little to no authority, into a reference to an object with more authority. File.Open is the quintessential example, where you can turn an immutable string that conveys no authority, into a file handle to your root file system!

File.Open is also typically accessible to all code, meaning it's also ambient authority. This classical file API is completely bonkers from a security perspective.

So we already have capabilities, what we really need to do is to stop adding all of this insecurity! The developers of the E language actually showed that this could be done by making a capability secure subset of Java called Joe-E, which removed the parts of Java and the standard library that exposed ambient authority or rights amplification patterns. Most Java code could run unmodified.

And as for whether capability security will ever be adopted elsewhere, it already has been! WASM's WASI has capability security in its core design, because capability security is exactly what you need for good isolation and virtualization, which are the core properties WASM needs.

> Source control? What's that?

I think squeak had Monticello for source control with their image based approach almost 20+ years ago and there was something else for smalltalk in the '80s too.

But yeah people like text and hate images, and I believe Pharo switched back to some git integration.

Smalltalk implementations have had text export/import for ages, and image based source control as you point out, is also quite old, Monticello wasn't the first.

In some way, Smalltalk had/has a more advanced and semantic diffing called ChangeSet

For example, CUIS book, 9.3 The Change Set:

https://cuis-smalltalk.github.io/TheCuisBook/The-Change-Set....

> something else for smalltalk

"ENVY/Manager augments this model by providing configuration management and version control facilities. All code is stored in a central database rather than in files associated with a particular image. Developers are continuously connected to this database; therefore changes are immediately visible to all developers."

https://www.google.com/books/edition/Mastering_ENVY_Develope...

~

1992 Product Review: Object Technology’s ENVY Developer

http://archive.esug.org/HistoricalDocuments/TheSmalltalkRepo...