Why we need Lisp machines
fultonsramblings.substack.com
fultonsramblings.substack.com
Today, we have computers that run Lisp orders of magnitude faster than any of those Lisp machines. And we have about 3–4 orders of magnitude more memory with 64-bits of integer and floating point goodness. And Lisp is touted to have remained one of the most powerful programming languages (I think it's true, but don't read into it too much).
Yet, it appears the median and mean Lisp programmer is producing Yet Another (TM) test framework, anaphoric macro library, utility library, syntactic quirk, or half-baked binding library to scratch an itch. Our Lisp programming environments are less than what they were in the 80s because everybody feels the current situation with SLIME and Emacs is good enough.
We don't "need" Lisp machines. We "need" Lisp software. What made a Lisp machines extraordinary wasn't the hardware, it was the software. Nothing today is impeding one from writing such software, except time, energy, interest, willpower, and/or money.
Don't get me wrong, there are some Lisp programmers today developing superlative libraries and applications [1], but the Lisp population is thin on them. I'd guess that the number of publicly known, interesting (by some metric), and maintained applications or libraries that have sprung up in the past decade probably fits on one side of a 3"x5" index card. [2]
Though I won't accuse the article's author of such, sometimes, I find, in a strange way, that pining for the Lisp machines of yore is actually a sort of mental gymnastic to absolve one for not having written anything interesting in Lisp, and to excuse one from ever being able to do so.
[1] Just to cherry-pick a recent example, Kandria is a neat platformer developed entirely in Common Lisp by an indie game studio, with a demo shipping on Steam: https://store.steampowered.com/app/1261430/Kandria/
[2] This doesn't mean there aren't enough foundational libraries, or "batteries", in Lisp. Though imperfect, this is by and large not an issue in 2022.
Discussed here https://news.ycombinator.com/item?id=30800520 The main issue is that Lisp, for all its inherent "power", has very limited tools for enforcing modularity boundaries in code and "programming in the large". So everything ends up being a bespoke solo-programmer project, there is no real shared development. You can see the modern GC-based/"managed" languages, perhaps most notably with Java, as Lisps that avoided this significant pitfall. This might explain much of their ongoing success.
Nowadays, there are large applications written by groups/teams, which are worked on for three or more decades. For example the ACL2 theorem prover has its roots in the 70s and is used/maintained until today for users in the chip industry.
Common Lisp was especially designed for the development and production use of complex programs. The military at that time wanted to have a single Lisp dialect for those - often applications came with their own Lisp, which made deployment difficult. A standard language for projects was then required.
These were usually developed by teams in the range of 5 - 100 people. Larger teams are rare.
Maybe not OSS, but somebody is keeping Lispworks and Franz in business.
It doesn't really take many contracts to scrap together office park rent and a staff of a few people. We're talking like < $1 million/year in revenue.
Good on them. Not every company needs to try to be the next Amazon
I know Lisp enough to have written programs of a few thousand lines in it. I'm not even slightly fazed by functional programming and (tail-)recursion instead of loops. I've read Steele's Common Lisp book from cover to cover. Someone even tried to get me to interview for a job writing Lisp (I politely told them I thought their system could not practically be implemented in Lisp and was, several years and tens of millions of dollars later, eventually proven right).
And I don't think the language has any redeeming features other than garbage collection and documentation, neither of which is notable in 2022. I'm someone familiar with the language who could quickly become productive in any decent Lisp, and that's what I think of Lisp. Can you imagine what a person new to the forest of parentheses, weird identifiers and rejection of 500 years of operator precedence notation thinks?
I'm curious to what that was, it sounds like a challenge more than anything.
I'm off a younger generation of programmers, and I've recently found lisp after coming from python. I still adore python and I can understand python's success, but I also adore lisp and its syntax, and I feel like I get the "powerful" argument for it.
For me, for what it's worth, I think the difficulties for lisp to join mainstream is really the thousands of little differences and gotchas that there's no single standard body looking after the users of the language. Princ instead of print, mapcars everywhere, etc. Trivial examples sure, but there's a lot that prevented me from making a few programs without doing through a book.
I'm not saying it's wrong, or it should be such and such a way. I am just simply comparing it to python and pointing to the first big elephant I come across.
Fwiw I also think that clojure is a lang that is capable of capturing more love because it's capable of tying in with Java ecosystem.. but it java is old and slow*; jobs in Java are slowing down, and clojure isn't filling in the gap, kotlin might be.
I take common lisp with me to holidays, because I like to learn it and all the funny words for things, but by far the lisp I use the most in day to day stuff is fennel, which is a lisp that compiles to Lua, and allows me to write plugins for neovim as well as a few other things that uses Lua.
TLDR: yes lisp needs software to be popular but it's not just a case of "make more applications" nor is the problem parents or operator precedence. People like sensibly named keywords.
I personally think things would be a little better if there was a CL to JavaScript transpilier, then you'd start to get some of the js crowd, same with python.
The thing is, developing maintained, maintainable projects is work, and everything I see about Lisp seems to have an undertone of being done, to some degree, for fun. It’s a language that scratches a specific itch; people feel clever about writing elegant code in it.
But the sad truth is that 99% of real application code doesn’t need to be clever or elegant. It needs to be simple and cheap and maintainable by anyone.
I do algorithm design, a lot of which is seeing what's possible/feasible before committing. I prototyped a classical AI planning system for orchestrating maintenance. there's no battle-tested standard library to reach for here. I needed prolog, I chose picat because it had a good planning engine.
before this, I worked on an NP-hard graph problem that involved tons of topological transformations for heuristics. I wrote the damn thing in C#. it was a nightmare. it's so hard to iterate and experiment in a language like that. even using something like Python/Jupyter, nested loops and horrible messes of bidirectional maps and indices bog you down. I should have done functional. nobody touched my code for two years because it was pure math. I think using C# made it much harder to understand - too much scaffolding obscured the semantics.
now I'm doing another NP-hard graph problem. this time I'm using MiniZinc. because it's concise, because it's powerful enough that I can at least figure out how to specify my problem and see where scale falls down.
so I'm in this position of using a menagerie of bizarre languages not because I'm a hipster obsessed with pretty code, but because my problem domains are really weird and maintainability doesn't matter when you're not sure if what your goal is even possible. something like Lisp - a lingua franca for weird DSLs - would be great, for my very niche use case.
Do you stick with the same (maybe-)esoteric language for the production implementation? And if so, does this then drag the rest of the production implementation into that language as well, or do you do the hard bit in one language and the routine stuff in another more common language?
I don't see any mention of "modular" or "boundaries" in the post you linked, so I'm assuming that it doesn't add extra context to your point.
You say "very limited tools for enforcing modularity boundaries", which I'm going to assume means that you believe that Lisps have constructs for creating modularity boundaries (e.g. unexported symbols in Common Lisp), and just don't enforce them (e.g. I can still use an unexported symbol in CL by writing foo::bar), in which case - I don't think that this is actually an issue.
Programmers are capable of doing all kinds of bad things with their code, and shooting themselves in the foot, yet I've never seen an indication that the ability to shoot yourself in the foot with a language noticeably contributes to its popularity (see: C, C++, Perl, Ruby).
Moreover, specializing to Common Lisp, it's not like CL allows you to accidentally access an unexported symbol in a package - you have to either deliberately use :: (which is bad style, and takes more effort than typing :) or you get a hard crash. This is opposed to the above listed languages, which allow you to shoot yourself in the foot in a large number of extremely diverse and interesting manners, often without giving you advance warning - and yet are still far more successful.
------------
I don't believe that the lack of success of Lisps are due to technical deficiencies.
I would also add some minor incidental technical factors (e.g. early Lisp implementations were far less efficient than C implementation during the time when efficiency really mattered, and then cultural/marketing factors kept C as the dominant language far after the technical balance tilted in favor of Lisp).
But this is not a technical deficiency of Lisp, it's more of a technical deficiency of humans.
One is not looking at dead code, instead one is working with it interactively.
Lisp is not brutalist concrete, Lisp is wet clay in the programmer's hands.
The main way to program software in Lisp is extending it while it is running and with the development tools as much integrated as possible. Thus a Lisp program has maximum information about itself, there is no separate debug mode, no long build times, no static code.
Here are some traditional notations found in Common Lisp:
"string"
symbol
:keyword-sym
#:gensym
package:sym
`(static content ,inserted ,@spliced)
(l i s t)
(im pro per li . st)
#(v e c t o r)
6.18e+23
13/17
;; comment
#*100111101101
#1=(1 2 3 #1#) ;; circular structure
#S(point :x 35 :y 44)
Lisp is not just (this (and that)).
This definitely doesn't describe the semantics of most Lisps, which operate at the higher levels of abstraction and conceal many implementation and hardware details from you by default.
It also doesn't describe the syntax of most Lisps, including all of the major ones. Common Lisp, in particular, has extensive syntactic sugar, as shown in kazinator's post.
> structural elements over decorative design
I mean, Lisps don't usually go out of their way to look "pretty", but neither do they actively attempt to look ugly. Most Lisps' syntax has had design effort put into it - the authors strive for consistency and ergonomics.
Taste is subjective, and I personally strongly prefer the syntax of Common Lisp to C or Python - both in terms of aesthetic appearance, and in terms of ergonomics.
Furthermore, very few programming languages (excluding most Lisps and all major languages) are intentionally designed to be "decorative". Whitespace, for instance, is added into languages for readability and parsing determinism, not beauty or decorative appeal.
I don't think that this metaphor is accurate at all. There are many criticisms that you can make of both the Lisp family and specifically of Common Lisp, but I don't think that "brutalist" is a valid one.
I think I understand why certain people really like Lisp -- I have enough experience with enough programming languages to understand the appeal. But I also believe it's unappealing to the majority and I don't think that's a controversial stance. It's not a simple matter of syntax -- I don't think it can be fundamentally still be Lisp and be broadly appealing.
Mainly, Lisp was developed on institutional hardware in the era of big iron. It became larger and more complicated, keeping up pace with the capabilities of those machines, which Lisp could easily strain.
Suddenly, microcomputers showed up in the 1970's. The industrial strength Lisps just wouldn't go down from hundreds of kilobytes or megabytes of RAM required down to two-digit kilobytes.
The situation didn't start to improve until probably the late 1980's.
Some developers used Lisp for prototyping, and had the expensive hardware; then to create a product to ship to users on consumer hardware, they rewrote the prototype. That story was exemplified by the CLIPS expert system.
An entire generation of programmers who started on microcomputers in the 1970's and 1980's wouldn't have even had opportunities to use Lisp.
When I started programming in 1982 or so, I inherited a stack of Creative Computing magazine back issues, so I read about all sorts of interesting stuff, including Lisp; but I wouldn't have been able to get my hands on it anywhere: and not for the lack of knowing about it or wanting.
Microcomputers steam-rolled a vast amount of software that preceded them. Software that didn't make the jump from big iron to microcomputers either died or was relegated to niches. Fortran, Cobol, operating systems like Multics, you name it.
We use Unix-like systems today because Unix started on minicomputers that were close in capability to microcomputers. There was only a small lag time between Unix being "bigger iron only" to running on microcomputers. It used pretty modest resources; e.g in 1980, /vmunix was maybe 50-something kilobytes. But even that was a little bit large, and the microcomputer jump did hurt Unix; it allowed disk operating systems like CP/M, PC-DOS and MS-DOS to gain a foothold. Unix never quite recovered; we now have free versions that are quite popular, but mostly in serving and embedded.
Speaking of Unix, I read a book on using Unix in the mid 1980's. As a kid. I had nowhere to actually access it for hands-on experience, but was exciting to read about. You had to be a university student, or else a grown-up working at some company or institution. (Or else, dial into something and break in.)
Anyways, the time lag between the start of the microcomputer revolution, and Lisp being able to run well like on 1970 big iron, was quite long! Long enough to see people retire. There is nobody out there to do mentoring. If millions of programmers started to be interested in Lisp, there would be nobody available for mentoring. Everyone would have to use just books and such. You can hardly find a course to take or anything like that. Some universities use some Lisp or Scheme dialect in a few courses, and that's about it.
Even if every great Lisp hacker who ever lived was still alive and interested in mentoring, it wouldn't be enough; there simply weren't that many people in computing 30, 40, 50 years ago.
Lisp is more capable now than any time in it's history and if you wanted to learn about it then there's also no better time in history to do that than right now.
I agree that there is no better time than now, but at the same time, the ratio of Lisp experts to total developers may also be at an all-time low.
And Lisp has experts! You're literally posting this to a massively popular forum for developers and it's written in a Lisp dialect!
Reddit, arguably one of the largest sites on the Internet, was written in Common Lisp before being re-written in Python.
Python is, semantically, pretty similar to Common Lisp - they have far more in Common than they do with C++, for instance. So, even if it's not the syntax, it's clearly not the semantics or language behavior, either.
I'm pretty sure it's the syntax, though. You used the word "unappealing", which often refers to a surface attraction or draw. I think that this is absolutely correct - Lisps are unpopular in large part to their very weird syntax, and indeed, one of the most common comments that you'll hear when you bring up Lisp is "Is that the weird one with all of the parentheses?"
This is further reinforced by the fact that most programmers have never even tried to run a single line of a Lisp - and the vast majority haven't invested enough time to learn enough to write a non-trivial program with it, and therefore haven't had enough exposure to even judge whether they like the language or not. Therefore, any reason for unpopularity must be superficial.
- none of your friends or coworkers use it.
- there are few jobs requiring it.
Basically, you can mock it all you want with practically zero social risk.
Jokes that target people of some specific nationality or regional group are not as tolerated any more nowadays, perhaps depending on the social company, but this is like those jokes. For instance "dumb Polack" jokes.
You've heard the one: "Frederic Chopin and Nicolaus Copernicus walk into a bar, immediately bespying Marie Curie sitting by herself. Stanislaw Lem is tending bar, pouring her a drink. ..."
in broadstrokes, imo, Lisp's endless plasticity of infinitely nested parentheses is both its greatest strength and... its greatest weakness
I love it at Write time. hate it at Read/Maintain time. why I've avoided it for serious work. Python, Java, Go Rust etc are all easier for my brain & eyes to quickly parse on the screen
which is unfortunate. because I still drool at Lisp's linguistic power
This readability loss does counteract the ability to reload forms into a running core. Especially now when i am trying so hard to make everything running be immutable and derived from various git repos.
Similarly, if you're working by yourself, or in a team with shared norms, you care far less about average function written by a random smart Lisp/Haskell/C++/Perl programmer, and more about what (a) you/your team writes and (b) the ergonomics of the language itself.
In fact, I argue that Lisp is slightly better than the above languages because much of its fanciness comes from macros, which you can trivially expand inline to simplify the code - which you can't do with fancy category theory constructs in Haskell, and is only barely possible with boost:: templates in C++, for instance.
(that isn't to say that I disagree with your point "the average function written by a smart lisp programmer..." - I completely agree with your assessment, I just don't think it's either as big of a problem as it could be, nor is it unique to Lisps)
An interesting perspective. From my POV, it's hard to think of a less Lisp-like language than Java. COBOL, maybe.
Building an environment that strongly and uncompromisingly expresses this philosophy at its core is a serious undertaking in terms of time investment. Emacs has been in continuous development for 37 years and while it is still not as good as Genera, it's certainly "good enough" for lots of people and definitely captures the spark of this philosophy.
In the Common Lisp world, we've had plenty of tries (MCL, Hemlock, McCLIM) but they've all failed to get people to coalesce and generate progresss.
Maybe the fact that Emacs exists is a curse in that people realize the barrier they'll have to clear to make something substantially better and decide to devote their energies into more easily realizable efforts.
People sometimes claim CL+Emacs+Slime provides the full interactive experience. I just can't agree with that at all. I have tried, and the experience was not substantially different from Go development and development in any other fast-compiling language with Emacs. In some respects, it's even worse than with various modern languages, even though most of those languages are strictly inferior to CL from a pure language perspective. If editing files in Emacs is all there is to the allegedly great Lisp experience, and developers at the same time have to deal with all those idiosyncrasies of CL such as CL's filesystem path handling, ASDF, and tons of poorly documented libraries, then I can't really see the advantages of CL. The language is powerful, don't get me wrong, but a truly interactive experience is totally different. Smalltalk managed to keep this experience but for some reason the Lisp community seems too have lost this vision. I guess the developer community is just not large enough.
Anyway, before someone tries to build another "close to metal" Lisp machine or tries to revive any old Lisp machine, I'd rather wish the community would focus on creating truly integrated development environments that abstract away from the host system and are fully hackable and editable from the ground up while maintaining security and multi-user access. A "virtual Lisp" machine with virtual OS, so to say. If that's developed for a certain purpose like building and deploying extremely portable web applications, I believe it can have a great future.
Sorry for the long rant. This is just my impression after having programmed in various Lisp dialects for the past three decades.
Emacs with Emacs Lisp on the other hand offers a great interactive experience that also manages to easily surpass every modern Smalltalk incarnation in practicality and size of development community. So if running Genera isn't easily doable, this will give you a taste of what Lisp interactivity is all about.
I think the (a?) reason for that is a (otherwise good) shift to production being immutable. When you aren't allowed to interact and change code in a running system, you lose a massive advantage of CL over simpler languages. When the answer to every error is "send a 4xx or 5xx to the client" then having a powerful error-handling system is similarly pointless. When you only write CRUD programs like everyone else, you're just plugging together other's libraries and not creating novel techniques or fancy math. In this world all CL's advantages are negated.
- Debugger is always ON.
- I can inspect the data I'm working with.
- I can redefine things without starting everything all over, avoid losing current context. Fast restart is not the same.
- I can evaluate pieces of code without the need of a REPL. Point to an s-expression and evaluate that piece of code, inspect the result.
I don't see how Smalltalk is much more interactive. It is more powerful at graphics and tools integration, but SLIME provides an interactive enough experience IMO, and it is significantly better to any fast compiling + restart language.
Nobody goes into Java because their self identity is "a Java programmer" to gather a team of people to create a Java machine running Java software to unleash the power of Java for the masses by enabling them to do everything in Java for the sake of doing everything in Java, By Java, With Java, For Java. And if they do talk like that they would be a Sun Microsystems marketing pamphlet from 1999, or a joking reference to Zombo.com, or suspected of having zero interesting ideas and defaulting to Ouroboros-naval-gazing.
Adobe Photoshop Lightroom is C++ and Lua. Blender is C++ and Python. Excel is C++ and Visual Basic for Applications. LibreOffice Calc is C++ and Python. These are large, popular, programmable systems which exist today and are good enough; good enough for people to spend lots of money on them, good enough for people to spend years skilling up in them, good enough that once they existed people wanted them to keep existing and they haven't faded into the past.
The added allure of an imaginary rebuild of them like "if you took the lid off Excel you'd see VBA inside so you could rework the way it handles multiple sheets using only a skill you have and software design skills and Excel-internals knowledge you don't have" would get a hearty side-eye and slowly backing away from most Excel users. "Everything inside looks the same" is as attractive as "if you open your car trunk you'll see leather seats and plastic dashboard components making it move" or "if you sit in this car you're sitting on engine parts because the top priority is that a welder in a scrapyard can build the entire car without leaving their comfort zone". There are certainly people who want that, but the way the world hasn't developed that way suggests it isn't particularly desirable. Even when such things have been built, people can today use a Smalltalk, an APL, save their running work in a memory-dump and reload it and rewrite parts of it in itself, people flocked to Jupyter notebooks instead.
> "[1] Kandria is a neat platformer developed entirely in Common Lisp"
https://cdn.akamai.steamstatic.com/steam/apps/1261430/ss_a3f...
Without mocking a team who has built, developed, polished and planned to release a project, because that is respectable, it also looks like computer gaming of the early to mid 1990s Intel 386/486 era; remeniscent of Prince of Persia, Gods, Lemmings, Monkey Island. But it needs an Intel i5, 4GB RAM and 1GB storage. It's not even released yet and has no reviews, but you describe it as 'superlative' ("Of the highest order, quality, or degree; surpassing or superior to all others") - are you rating it so highly based on it being written in Lisp or what?
As for Kandria, did you play the demo, or did you just look at screenshots and system requirements and make your brazen judgment? I don't think Kandria is at all amateur or sloppy, regardless of to which aesthetic era you think it belongs. Many have claimed that it's not even possible to write a game that doesn't blow because Lisp is dynamically typed and garbage collected. Now the goalposts have moved to, "well, it takes 1GB of memory and doesn't even look like it's from 2022."
I commend anybody who ships.
> "Many have claimed that it's not even possible to write a game that doesn't blow because Lisp is dynamically typed and garbage collected. Now the goalposts have moved to, "well, it takes 1GB of memory and doesn't even look like it's from 2022.""
JavaScript appeared, took over the world, demonstrated good games in a dynamically typed and garbage collected language at least a decade ago. Goalposts move, time moves on. Some AI people complain "once you wanted AI to beat chess, now that's not good enough? Stop moving goalposts!". Software today can recognise people, generate images from text descriptions, complete sentences, describe photos, self-drive cars over constrained environments, walk robots over rough terrain and jump onto ledges, steady cameras on flying drones following a person. DeepBlue beating Kasparov was impressive in 1996, it's not impressive now. There are AI experts today who were born after that.
Especially contrasted with "it's the best language", "superlative applications", linking something which looks like software of 25 years ago on 10,000x less powerful computers is a big difference in expectations. (It may actually be an amazing game, hence me asking what it was that made you say it is, before it's released). Years ago a company writing Transport Tycoon in assembler was very impressive. Now a single person can write a math animation video generator in Python as a hobby side-project while being a double-major student.[1] Expectations ramp up, year on year, and "coding on the libraries of giants" is a real effect. Pythagoras calculating the length of a hypotenuse was impressive. A school student doing it today isn't.
Hacker News is written in Arc. It's impressive to build a language and build a forum in that language, even though forums existed years ago. But if someone claimed it was the best language which needs to be preserved because it can do the best things, and then used HN as the example, anyone who has used a modern forum with all the trimmings would do the "yes, Grandpa, everything was better in the old days" polite smiling and nodding.
Electron isn't bad because it lacks Lisp, it's bad because it's sluggish and ramps up fans and drains battery life. WhatsApp isn't great because it was written in Erlang, it's great because it connected hundreds of millions of people on all kinds of featurephones and early smartphones. Visual Studio Code isn't very customisable because it's written in JavaScript, it's customisable because they built it to have lots of extension points. The answer to why we need software written in Lisp is like the answer to why we need software written in APL or Prolog: we don't. We also don't need softwre written in Java or C# or Python or Ruby. We may need software written in C, x64 Assembler, because of hardware lock-in. Tools are for doing things with, not for falling in love with.
In isolation and without context: We do not need software written in Lisp. Nothing compels us to choose Lisp as a language to express programs that solve problems.
If we want a non-UNIX, non-C ecosystem, then what we need is software, not hardware. Lisp is one type of previously proven ecosystem that works. So it is reasonable to discuss that as a potential option, which is partly the topic of the article we both are posting comments to. I argue that if we want a Lisp ecosystem, we need better and more comprehensive software written in Lisp.
If we don't want a non-UNIX, non-C ecosystem, then we needn't discuss writing software in Lisp (or Smalltalk or ...) as a possible solution to that (non-)problem.
In any case, I simply argue that old hardware or even obsolete operating systems aren't really a productive thing to talk about in this context, except on a case-by-case basis.
Again, regarding "superlative", the word is in reference to last-decade Lisp software alone, which I contend describes Kandria. I don't even attempt to compare software written in Lisp to the entire universe and history of software, which it seems you're doing, but doesn't seem pertinent to the discussion. (Though, to be clear, even when comparing to all of that, I still think you're wrong. But we can argue to no end about our own subjective opinions.)
HN could have indeed been written in many languages, but IMO the simplicity is a strength. I'd rather be here than twitter, reddit, or any other forum. There's no dark patterns, no recommendation engines or other annoying nonsense trying to game "engagement", no ads, basically none of the things that make the modern Internet an awful user experience.
You might have heard of this thing called "Art", and that it has styles, and that not only one of them is called "pixel art" for celebrating that kind of limitations, Art as a whole is often talked in terms of self-imposed limits used in creation of a work.
That said, a game can deliberately target such style, and yet hide considerable richness of implementation (consider: Noita, Dwarf Fortress).
Another thing with Shinmera and his team is that they produce both interesting stories, interesting games, but also code I'd argue is art too.
It definitely can, and Monkey Island and other SCUMMVM games are examples of having lots of gameplay and comedy, despite constrained graphics. As are card games and board games, for that matter. Fancy 3D isn't the be-all, end-all. In more recent years SpeedRunner[1] isn't pixel art but it's quite simply styled, and is fun for leaning so heavily on a single game mechanic.
I'm not meaning to diss Kandria which might very well be a great game. I meant to call out the gulf between Lisp as "the best language" and "superlative applications" being developed with it then linking to Kandria might set one up to think of the "best games" of recent years, by popularity or profitability or ambitiousness or multiplayability, or replayability, or storyline, or VR support, e.g. Pokemon Go, Grand Theft Auto series, Dark Souls series, Fortnite, Fifa, Half Life Alyx, Tony Hawks Pro Skater remakes, Spiderman PS5, Elite Dangerous, Roblox and Minecraft, Final Fantasy series. Which all seem to have done alright without Lisp, and no game development houses have done the Paul Graham "Lisp as secret weapon" to make a game nobody else can make with other tools.
[1] https://steamuserimages-a.akamaihd.net/ugc/58020036161797293...
As for games where Lisp was secret sauce, Uncharted series would count, I think ;) It was a return to GOOL-style programming (like in Crash Bandicoot) instead of full engine written in Lisp (like GOAL was for Jak&Dexter), however for narrative-heavy game the tooling used for making that narrative should count as secret weapon.
BTW, a huge underappreciated thing about why Kandria can't fit into less RAM is simple fact that today games generally render at much higher resolutions, and I believe Kandria does have multiple buffers and passes involved (like most game rendering engines these days, though I will have to read Shinmera's paper about shader composition system to verify)
Ethereum was a mistake.
They should just move to their own chain entirely. Maybe this is a baby step towards that. Each Galaxy/Star is already basically a staker in a Proof of Stake network.
galaxies/stars can just charge for services provides like an ISP basically
urbit to me is exactly the opposite
Got any examples?
I wonder what the lisp os guy are thinking about OS / UI these days.
I don't think this is true. Not anywhere close. Most such examples are small, and probably only took a small number of hours to produce. While "superlative" stuff takes very many man hours to create. So just by seeing that there are many throwaway testing frameworks or whatever, you cannot tell where most of the work hours actually go. A half baked binding library takes 20 minutes to make, while a proper high quality rendering engine takes hundreds if not thousands of hours.
The Lisp population is thin on people making cool shit because the Lisp population in general is thin.
In fact I conjecture that this is the reason Unix is more popular than Lisp -- because Lisps don't interoperate well. They haven't built up a big ecosystem of reusable code.
Whereas Python, JavaScript, R, C, C++, and Rust programmers can reuse each others' code via Unix-style coarse-grained composition. (Not just pipes -- think about a web server running behind nginx, or git reusing SSH and HTTP as transports.)
You can also use link time composition. It takes some work but it's better than rewriting your Common Lisp code from scratch in Clojure.
-----
Honest question: how do you communicate between two Lisp processes on two different machines? I know Clojure has EDN (which is sort of like JSON : JavaScript), but I haven't heard of the solutions for other Lisps.
I wrote about this problem here: A Sketch of the Biggest Idea in Software Architecture http://www.oilshell.org/blog/2022/03/backlog-arch.html
> The lowest common denominator between a Common Lisp, Clojure, and Racket program is a Bourne shell script (and eventually an Oil script).
I'll definitely update it if there's something I'm missing.
I would say the design of Unix is "rotting", but the answer is to IMPROVE Unix. Not dream of clean slate designs that will never be deployed. Plus this post doesn't actually propose anything. If you actually start trying to build your Lisp machine, I believe you will run into dozens of reasons why it's not a good idea.
Clojure is something else entirely which is why a lot of people don't consider it a Lisp.
> Honest question: how do you communicate between two Lisp processes on two different machines?
If you want to use built-in object serialization, there is print and read.
Could you say more about what you mean by this? Is there another family of Lisps that excludes these three? I've met people who make a big deal about lisp-1 vs lisp-2 (https://en.wikipedia.org/wiki/Lisp-1_vs._Lisp-2), and which is the right way to be a Lisp, but I think maybe those people just enjoy being pedantic.
Probably TCP or UDP based protocols like essentially every cross-network communication in every language today.
EDIT: Also, it should be noted that JSON does not, itself, allow you to communicate across a network. It's just a serialization format. You still have to select some protocol for actually doing the communication. If your answer to the question "How do you communicate between two JavaScript processes on two different machines?" is "JSON", you've failed at actually answering the question.
But I keep having this conversation with people who are mad at Unix (and Unix shell) and searching for an alternative. Some of them are just fantasizing, as with this post, but some of them are actively working in that direction.
One possible alternative is if the kernel grew a mechanism to pass linked data structures directly, instead of requiring serialization and parsing (a little like Go channels perhaps). I suppose you can kind of do that with shared memory -- but not really because copying is roughly equivalent to serialization.
This might sound like an unrealistic idea, but people are seriously arguing for it. And of course they always argue in favor of their favorite language's data structures, not realizing it comes up with a boatload of design choices specific a problem domain, and that other problems require different data structures.
This post is arguing for something even more radical: an entire machine in Lisp, with a single address space (really bad idea).
----
The other more practical thing I'm arguing against in the post is a certain design of alternative Unix shell [1] that I've been seeing a lot.
The design is that basically that all the tools like 'ls' and 'rm' and even networking tools are expected to be statically linked into the shell binary.
And many of them pass structured data directly, specifically to get rid of serialization and parsing. There are at least 3 or 4 shells with this design.
My point is that this creates a "two-tiered" shell design and that composition is inhibited. In my mind the point of the shell, and the power of it, is to be situated within or "woven into" the OS. It interacts with everything on the system. It's not a "world" unto its own. You should be able to whip up a script in your favorite language and call it from your shell, and compose it with other tools you didn't write. You shouldn't be forced to "escape" the shell to call outside tools, or write algorithms in shell like you would write algorithms in Python or C. The shell is for coarse-grained composition and coordination.
[1] list here: https://github.com/oilshell/oil/wiki/Alternative-Shells
----
Another possible alternative is something like https://www.unison-lang.org/ where source code is not text; it's structured data. This has big consequences that I hinted at in the prior post, and have in my notes for upcoming posts:
They are writing their own version control system and editor because they don't use text. (I think experiments like this are interesting; it's definitely worth some portion of our efforts.) It is true that they have to use byte streams at the end of the day because they have to use storage and networking hardware, but the user is supposedly insulated from any of that. That is, as long as you stay in this hypothetical universe. Again I will bring up interop as a big problem.
A lot of people seem to imagine "monoglot" worlds where they wrote everything from scratch and no other code exists. Everything done in the past must be thrown out because we know better now. :)
Elm is a more widely used language with similar interop problems.
Most higher level languages I've worked with seem to focus on using .so files rather than producing them.
This means the lowest common denominator for the unix ecosystem is what C can provide. Otherwise stated, unix is marching forward at the pace of C.
In fact, Symbolics shipped compilers for other non-Lisp programming languages, including C and Ada. These interoperated smoothly with Lisp code, much more so than they do under Unix. In this demo, Kalman Reti compiles a JPEG decoder written in C, and replaces the Lisp JPEG decoder that came with the system with it, yielding a performance boost:
I still think various Lisps don't interoperate enough today, but I'm not very familiar with the Lisp machines of the past. If it can interoperate with C and Ada that's interesting. But I also wonder about interop with JavaScript :) i.e. not just existing languages but FUTURE languages.
These are the M x N problems and extensibility problems I'm talking about on the blog.
Or parenscript: https://parenscript.common-lisp.dev/
Or CLOG: https://github.com/rabbibotton/clog
And, there’s always the option of writing an HTTP or websocket server and serving JSON.
I personally use the SLIME repl in emacs for a ton of things I used to use bash one-liners for and you can push it pretty far (this is an experiment, I’ve wrapped it up more nicely into my .sbclrc since):
https://twitter.com/fwoaroof/status/1502881957012668417?s=21...
It'd be more accurate to say "The lowest common denominator between a Common Lisp, Clojure, and Racket program is sexpr notation." By using sexprs over a pipe, named pipe, or network socket, you can very easily get any of those Lisps to intercommunicate deeply structured data with any other. This is how SLIME and its SWANK protocol work. I don't even think the shell is involved; Emacs handles spawning the inferior Lisp and setting up the communication channel itself.
The thing the Lisp machines had was a very robust ABI. Lisp has a specific notion about what a function is in memory. This is largely because Lisp is always "on" on a Lisp machine, there is no point at which you cannot take advantage of the entire Lisp runtime, including the compiler, in your own programs. Accordingly the Lisp machine C compiler output Lisp functions containing code compiled from C, that could be called by Lisp functions directly (and vice versa). Presumably a JavaScript runtime for Lisp machines would be able to do the same thing.
By contrast, C has no notion of what a function is; and the C ABI used by many operating systems presents a function as simply a pointer in memory that gets called after some parameters are pushed to the stack or left in registers. (How many parameters, their type, and their size, is unspecified and simply agreed upon by caller and callee.) Nothing about memory management is provided by the runtime either. All that has to be provided by user programs. All this adds friction to interoperation by function call, and makes IPC the most straightforward way to interoperate.
But oh well. Our computers are all slowly turning into JavaScript machines anyway, so maybe those Lisp/Smalltalk happy days can return again soon.
If you're thinking that's tautological (how else would Common Lisp and Clojure communicate?), then this subthread might help with the context:
https://news.ycombinator.com/item?id=30814716
I don't think Common Lisp, Clojure, and Racket are compatible with Emacs' s-expression format. Lots of people here are saying they use JSON or the like with Lisp, not s-expressions.
Emacs can use its own format to communicate with itself, because it controls what process is on the other side of the wire.
(In any case, any variety of s-expressions IS TEXT; there are non-trivial issues to get it back in memory -- like what to do about graphs and sharing, node cycles, integer sizes, string encodings, etc.)
But the point of the slogan is that when you have even a small bit of heterogeneity (among Lisps) then what you're left with is "Unix". A Lisp machine isn't a big win for this reason.
It is cool that Lisp machines had a robust ABI. That could solve the IPC problem. But then you have the problem of a Lisp machine communicating with a Unix machine, which I'm sure was a very real thing back in the day. So now you're left with the lowest common denominator of Unix. Again that is coarse-grained composition over wires, which is analogous to shell. The shell doesn't have to be involved strictly speaking, but the syscalls that are made and parsing that is done is pretty much the same.
Not a problem - you don't need (or want) a single Lisp. A hypothetical Lisp OS would support a standard runtime and typed IPC system that host programs can use (a la Windows and COM, or dbus), and nothing prevents you from using your own custom communication protocol over pipes/sockets instead.
I don't agree with your implicit assertion that there is a single reason why Unix is more popular than Lisp (machines) - I think that there are a multitude of reasons. Certainly, the most impactful one isn't a lack of interoperability between Lisps - it would be something like the bandwagon effect, or the fact that Bell Labs gave away Unix to universities, or the inefficient implementations of early Lisps.
I'm also fairly confident that the lack of a Lisp ecosystem is not because of a particular lack of interoperability (after all, Python has a massive ecosystem that is built almost exclusively on (a) C FFI and (b) other Python code), but for cultural reasons - Lispers just don't like collaborating together, and enjoy reinventing the wheel. These tendencies have been documented in [1] and many other places, and are highly consistent with my own experience in the Common Lisp community.
> Honest question: how do you communicate between two Lisp processes on two different machines?
Use PRINT to serialize an object on one machine, ferry it over to another one through whatever means you wish, and then READ it on the other. Or: "in the same way that two processes in most other languages on two different machines communicate". Serialize, transport, deserialize. Yes, the ecosystem is far less mature, and you'll have to do more legwork, but fundamentally the process is the same as in, say, Python. (Erlang might be an exception and have language-lever support for this, I'm not sure)
This method works for different implementations of the same Lisp, and even for different Lisps, under some constraints (e.g. while a list with the integers 1, 2, and 3 is represented as the s-expression (1 2 3) in almost every Lisp you can find, CL represents "true" as T, while Scheme represents it as #true, so you'll have to work around that). If you will, you can just use JSON or XML as a serialization format instead - every non-trivial Lisp has libraries for those formats, and some have libraries for ASN.1, too.
>> The lowest common denominator between a Common Lisp, Clojure, and Racket program is a Bourne shell script (and eventually an Oil script).
All of those languages share basic s-expression syntax described above, which is rather higher-level than a Bourne shell script. Why do you say that the latter is the "lowest common denominator"?
For that matter, why do you exclude the idea that JSON or XML aren't "common denominators" between Lisp programs, or even between Lisps, Python, and C++?
----------------
Your article says "Text Is The Only Thing You Can Agree On", but "text" isn't even a single thing. The set of bytes allowed by ASCII vs UTF-8 vs UTF-16 aren't the same. Even if it was, plain text is purely sequential and flat. Program interoperability requires structure. If the structure isn't directly encoded in a given "substrate" (such as text), and you need another layer on top, then that substrate isn't actually the one providing the interoperability. You say "Text is the most structured format they all agree on" but text isn't even structured at all, and nobody completely agrees on it - "bytes" is the only thing that fits into place here (which is equivalent to saying that there's no structured communication method that all programs agree on, which is true).
Put another way - programs do not communicate using plain text. They communicate with either an ad-hoc protocol built on top of text (that still has a set of constraints that make it incompatible with other protocols built on top of text), or they use a standardized format like JSON or XML, or even something like ASN.1 that isn't a subset of text.
Communication using text does not make programs interoperable. Bytes might be a "narrow waist", but text is factually not - if it was, you wouldn't need to use sed/awk/perl one-liners to connect various Unix utilities to each other, because the very fact that they were all using text input/output would make them interoperable.
You say "Tables and documents are essential structures in software, and expressing them in JSON is awkward." but you can express them in JSON. You cannot express those structures in "plain text", because "plain text" is not capable of expressing structure at all, and the best you can do is build a protocol that is a subset of plain text that can express structure (JSON, XML, CSV, etc.)
------------
If anything, Lisps are more interoperable than Unix, because the "lowest common denominator" of Unix utilities is "plain text" (which by definition cannot encode structure), while the lowest common denominator of Lisps is some common subset of their s-expression formats, which can encode structure.
------------
> I would say the design of Unix is "rotting", but the answer is to IMPROVE Unix.
You say this, but you haven't given a reason for why that's the answer.
Here's a reason why improving Unix is not the answer: because "everything is text" is a fundamentally flawed paradigm that introduces needless complexity and fragility into the whole system design.
> Not dream of clean slate designs that will never be deployed.
You're mixing the normative and the positive - "Unix should be improved" with "clean-slate designs won't be deployed". Are you making an argument about what should happen, or what will happen? (there's no guarantee that any improvements to Unix will be deployed, either)
> Plus this post doesn't actually propose anything. If you actually start trying to build your Lisp machine, I believe you will run into dozens of reasons why it's not a good idea.
Oh, yes, and I can start with a few: first, there's good reason to believe that high-level CPUs are a bad idea (as suggested by the fact that nobody has been able to make an effective one[2]); second, the security-less design of older Lisp machines is a terrible idea (in a vacuum); third, that a machine that only runs Lisp is going to be unpopular and there's no single Lisp and Lisps are not the final stage of PL evolution.
...but the arguments made by the article as to why Unix is inadequate are still solid, even if the author is suggesting a suboptimal solution due to nostalgia.
[1] https://www.lambdassociates.org/blog/bipolar.htm [2] http://yosefk.com/blog/the-high-level-cpu-challenge.html
For that matter, why do you exclude the idea that JSON or XML aren't "common denominators" between Lisp programs, or even between Lisps, Python, and C++?
JSON is mentioned in the post as A narrow waist, not THE narrow waist (of an operating system or of the Internet). I also mention CSV and HTML as "on the same level".
Likewise, bytes and text have a similar hierarchical relationship. I mention that in the post and also show some diagrams in the previous post.
If anything, Lisps are more interoperable than Unix, because the "lowest common denominator" of Unix utilities is "plain text" (which by definition cannot encode structure), while the lowest common denominator of Lisps is some common subset of their s-expression formats, which can encode structure.
JSON, CSV, and HTML are all built on top of plain text. You can store them in source control and you can use grep on them, and you can build more specialized tools for them (which has been done multile times.)
What I'm contrasting this with is people who say that we should build s-expressions into the kernel -- i.e. passing linked data structures directly, rather than bytes/text.
See the threads linked in the posts -- a variant of this same argument came up. This is very related to the idea of building Lisp into hardware, which I view as a bad idea.
You say this, but you haven't given a reason for why that's the answer.
The article is analyzing why empirically Unix, the web, and the Internet have worked in multiple respects -- interoperability, generality, scalability in multiple dimensions and multiple orders of magnitude, extensibility over decades, the polyglot nature, etc.
These are obviously successful systems, and I think it is pretty crazy to have the opinion that because it makes you parse things that a design that tries to eliminate parsing much be better along many/all dimensions!
These kinds of (IMO naive) arguments are exactly why I wrote the last 2 posts. They were popular and highly commented upon, and most people got it, or at least appreciated the tradeoffs I highlighted.
So I don't need to convince everybody -- as I said in the intro to the post, the more important task is to build something that embodies these ideas. I look forward to your code and blog posts; I do think there is something to the top criticism in the thread: https://news.ycombinator.com/item?id=30812626
I would definitely appreciate you pointing out exactly where the misunderstandings are! (also, ironically, most of the points that you make here were already addressed in my comment, including refutations of some of the ideas in your articles)
> JSON is mentioned in the post as A narrow waist, not THE narrow waist (of an operating system or of the Internet). I also mention CSV and HTML as "on the same level".
I saw that JSON was described as a narrow waist; however, the specific context of my question was about the use of those technologies applied to Lisps - if you believed that those were narrow waists, then you wouldn't have written both of the following:
> The lowest common denominator between a Common Lisp, Clojure, and Racket program is a Bourne shell script (and eventually an Oil script).
> What Is a Narrow Waist? [...] Small, simple mechanisms like the Internet Protocol, UTF-8, and JSON.
That is, there's an inconsistency between two statements both made by you, not a misunderstanding on my part.
> JSON, CSV, and HTML are all built on top of plain text. You can store them in source control and you can use grep on them, and you can build more specialized tools for them (which has been done multile times.)
Yes, and as I said in my initial post, that's both incidental and irrelevant. You missed or ignored several paragraphs of my comment you're responding to (starting with "Your article says"). I'll re-state for the sake of concision: the fact that those technologies are built on top of plain text does not mean that plain text is a narrow waist! All of those formats require the same level of parsing effort as they would as a byte-oriented protocol that didn't adhere to the subset of "plain text". This was, again, already addressed in my comment.
> See the threads linked in the posts -- a variant of this same argument came up.
You say "the posts" but there are at least four different "zones" that you could be referring to (the other posts on the Lisp machine article, links in your blog posts, or the Reddit or HN posts on your blog posts), and dozens-to-hundreds of individual links/comments in each of those "zones". "the posts" means nothing - please either link to a specific web page (or part of it, if it's long), or re-state the argument in your own words, so I can see what your argument is.
> This is very related to the idea of building Lisp into hardware, which I view as a bad idea.
I'm not defending that idea - I also thing it's a bad idea! I'm arguing against the specific idea of "text" being a "narrow waist".
> The article is analyzing why empirically Unix, the web, and the Internet have worked in multiple respects
You say this, but I can find zero evidence in either of your posts to support the assertion that the success of those things is a direct result of their use of plain text as a narrow waist. Thing like HTML don't count, because again, the fact that HTML builds on top of plain text is incidental and irrelevant.
> These are obviously successful systems
...and there are dozens of reason why a system can be successful, many of them social/cultural and not technical. Why is Windows so popular? Gmail? C++? Perl? SQL? Java? x86? DVD/Bluray? JavaScript? Are you going to tell me that the popularity of all of these systems are mainly due to technical reasons, and not business, social, cultural, and incidental ones?
In the specific case of Java, I can tell you that the main reason for its success was because Sun bribed universities to teach their CS courses in Java in exchange for free Sun workstations, and then the students went out and spread Java around. This is a perfect counterexample to the extremely naive idea that "technical superiority leads to market dominance", and there are dozens more out there.
Actually, given that we're on HN, any successful startup founder will tell you that technical superiority will not make your company successful (and of course we can then translate that to success in the open-source world).
> I think it is pretty crazy to have the opinion that because it makes you parse things that a design that tries to eliminate parsing much be better along many/all dimensions!
Then I'm going to say that I think that it is pretty crazy to have the opinion that a system that requires you to completely unnecessarily add serialization and deserialization for virtually no benefit whatsoever (and many significant drawbacks) is somehow better overall than a system that...doesn't.
> These kinds of (IMO naive) arguments are exactly why I wrote the last 2 posts.
I'm coming up with concrete counterarguments to every single point that you concretely state. Calling them "naive" instead of addressing their content is...not an indicator of sound reasoning, to say the least.
> They were popular and highly commented upon, and most people got it, or at least appreciated the tradeoffs I highlighted.
And, of course, none of those things are indicators of sound reasoning, either - appeal to authority and popularity fallacies, and all that.
> So I don't need to convince everybody
Well, unless you make concrete and valid arguments, I won't be convinced - although you can always find people who are convinced without those...
> as I said in the intro to the post, the more important task is to build something that embodies these ideas.
I am building things that embody these ideas. They're likely never going to be open-source, though, so I'm not surprised if you don't believe me.
> I look forward to your code and blog posts
The idea that blog posts are somehow more authoritative or substantial than any kind of writing is...something, for sure.
Unless you're saying that a design paradigm isn't good unless it's implemented in software, in which case that's clearly false, given the number of completely terrible design paradigm that have been used to design "working" software.
Depends what level of integration you want: custom tcp/udp protocols, HTTP and websockets are all pretty easy. But, you can also use something like this over a trusted network/vpn: https://github.com/brown/swank-client
using continuations with sexps is insanely powerful
send (+ 1 3 (lambda (x) `(handle-response, x)))
Although you have no idea what I am talking about yet just wait a bit more :))
And why are there so many? IMO the language is too flexible for its own good. It promotes this curious intellectual solo-competition where you try to prove you are worth of the elite who get all the fancy FP stuff and make all this bespoke functionality.
It's almost impossible for Lisp to be popular. To be popular means a lot of people use it, but that means it can't be complicated much above the median. But because it lets individual programmers push intellectual boundaries it self-selects itself out of this pool. Any big collaborative project will attract this type of developer and soon the lesser developers (who don't dare object for fear of appearing dumb) are not able to contribute.
Just my opinion, if a little dramatic.
Same as every other language: you pick a protocol and use it on both sides. Many of us already have enough JSON in play that it makes sense to start there.
(3600-series Symbolics machines also had a 68k "front end processor", but no Unix port was provided for it; they also ultimately had a C compiler that could generate code for the "Lisp processor", but the code it generated was intended to run in the Lisp environment.)
It's also worth noting that systems-level code for Symbolics machines (and, I presume, LMI as well) made frequent use of "unsafe subprimitives", misuse of which could easily crash the machine. And, unfortunately, if you needed to, say, get anything close to hardware bandwidth out of the disk drives, some of this became well-nigh unavoidable, due to poor performance of the OS-level file system (LMFS).
Later Open Genera was sold as a Virtual Lisp Machine running on a DEC Alpha / UNIX system.
I think the big problem with Genera is the licensing. Although it comes with source code, it is proprietary software, and buying a license is expensive. I think the owners of the Symbolics IP have prioritised squeezing the maximum revenue out of a declining user base over trying to grow that user base.
I'm surprised "Open Source LispOS" projects have largely failed to gain traction. Writing your own OS is (at least in some ways) easier than it used to be (especially if you target virtualisation rather than bare metal). There seem to be a lot more people saying "LispOS is what we need!" than actually writing one or contributing to an existing effort to write one.
> UNIX isn’t good enough anymore and it’s getting worse
Why exactly?
> A new operating system means we can explore new ideas in new ways.
LISP machines were not only OSes but also hardware. Is the author also proposing running this OS on optimized hardware or simply using our x86-64/AMD/M1 CPUs?
> With lisp machines, we can cut out the complicated multi-language, multi library mess from the stack, eliminate memory leaks and questions of type safety, binary exploits, and millions of lines of sheer complexity that clog up modern computers.
Sure, but it also requires rewriting a lot of these things, introducing and fixing new bugs... It feels like the good ol' "let's rewrite this program" that quite frequently doesn't live up to the expectations [1].
[1] https://vibratingmelon.com/2011/06/10/why-you-should-almost-...
>Why exactly?
Personally? We're in a bit of a transition point, and a lot of the technologies aren't working together like they used to.
An example, on my laptop I want to run android apps. The way to do this that actually works well (waydroid) only supports wayland. Unfortunately I use x2x to control another display remotely, and x2x doesn't work properly under wayland, and never will due to wayland's security choices.
So like, what am I supposed to do here? Not run android apps? Not use tools like barrier/synergy/x2x?
This is one of many many frustrations I've had from this new generation of wayland/systemd/etc. Hopefully it gets better eventually but it does feel a lot like the rug is constantly being pulled out from under me for no good reason...
Now I don't think a lisp machine is going to fix that mind you, but it is a concern.
>That plus existing solutions for Wayland network transparency should be enough.
If you're saying it will fix my x2x use case, well it won't. Wayland's security model fundamentally prevents this use case. Maybe someone will add extensions to it eventually but right now the way compositors handle mouse capture seriously prevents this use case, and I'm skeptical that all the different compositors will agree on a solution any time in the next 10 years...
So I'm stuck using X on both displays for the foreseeable future if I want to use x2x/synergy like functionality, and I'm certain that it's going to become harder and harder to keep using X over time...
I have had ideas for some applications, but can’t do them because the libraries I need are written in different languages, which don’t interoperate well (and one I am not familiar in).
Every post here looking for recommendations has many responses with different packages/ecosystems doing the same thing.
Sometimes I feel like there are too many developers and not enough of them really interested in the actual hard problems. So they just make another python package manager or webapp framework.
I think a wave of new standards causes issues like the above, and eventually this gets worked out.
Not all "standards" in practice become standard. We learn a lot of "standards" were bad ideas, and they eventually get tossed out and most people forget anyone was ever trying to make that a thing.
I don't want a new language, or a new preprocesser, or a new way of thinking about programs... I just want a one click way to take a folder of HTML that looks like a static site, and package it up into cross platform apps with all the proper API access.
I don't care about cryptocurrency and global decentralized databases, I just want to be able to run a site by buying a NAS appliances, putting files on it, and sharing the QR code on it without any signups and cloud accounts.
The hard problems are mostly solved. There's probably some long dead random github repo that does anything you want. They're just not packaged for all platforms and maintained professionally.
I don't need a phone without any binary blobs at all... I just want a mesh network feature and a fair trade stamp.
Because... it sucks to maintain something novel even if it's all just pieced together from npm. It sucks to be the one responding to issues. It sucks to actually run a project and keep track of devops stuff.
All the tech has been invented dozens of times over, but never polished, and always ruined with some unnecessary immutable log feature or performance destroying random access low latency routing thing like IPFS has.
The former is focussed on tinkering, nostalgia, tool-building, wheel reinventions, and half-finished build-and-forget spare time projects.
The latter has successfully applied high-friction chokeholds around startup culture which makes the cost of entry for new business ideas far higher than it really needs to be.
There is almost no blue sky development equivalent to the original Internet, the Mother of All Demos, and the desktop computing model - and associated projects - created at PARC in the 70s and early 80s.
It's all very "mature", incremental, slow, and me-too.
And so expensive, both financially and in terms of developer time.
I'd love to see a renewed interest in attacking this from both ends - a movement to really push the imaginative limits of what's possible, while also being implacably humane and user-oriented and indifferent to ad tech, tinker tech, and pre-commercialised cloud tech.
There's just certain things they can't do because their incentive is stuff you constantly pay for all the time.
I'd love to be part of something like old 90s revolutionary software.
But I've noticed close to zero interest from the dev community in anything like that.
Everyone mostly just wants to play with sorting algorithms or write their own small language, unless they're getting paid a whole lot. It's become a sport more than an engineering discipline.
It's hard to imagine Excel or BitTorrent being written now.
Heck, it's hard to even imagine web browsers with JavaScript being written now.
What kind of projects would you like to see/work on?
the basic problem is fragmentation and assembling a community of interest.
Writing a package manager is hard in some sense, but you have an idea of what it needs to do, what features it should have, and what success ultimately looks like.
But there are problems, large and small, which require new thinking and leaps of faith. There are people working on these, of course, but sometimes I feel they are being neglected.
(and people working on those problems probably make 10-25% of what many FANG developers make...)
I envy you!
https://en.wikipedia.org/wiki/The_UNIX-HATERS_Handbook
https://web.mit.edu/~simsong/www/ugh.pdf
>The year was 1987, and Michael Travers, a graduate student at the MIT Media Laboratory, was taking his first steps into the future. For years Travers had written large and beautiful programs at the console of his Symbolics Lisp Machine (affectionately known as a LispM), one of two stateof-the-art AI workstations at the Lab. But it was all coming to an end. In the interest of cost and efficiency, the Media Lab had decided to purge its LispMs. If Travers wanted to continue doing research at MIT, he discovered, he would have to use the Lab’s VAX mainframe.
>The VAX ran Unix.
>MIT has a long tradition of mailing lists devoted to particular operating systems. These are lists for systems hackers, such as ITS-LOVERS, which was organized for programmers and users of the MIT Artificial Intelligence Laboratory’s Incompatible Timesharing System. These lists are for experts, for people who can—and have—written their own operating systems. Michael Travers decided to create a new list. He called it UNIXHATERS:
Date: Thu, 1 Oct 87 13:13:41 EDT
From: Michael Travers <mt>
To: UNIX-HATERS
Subject: Welcome to UNIX-HATERS
In the tradition of TWENEX-HATERS, a mailing list for surly folk
who have difficulty accepting the latest in operating system technology.
If you are not in fact a Unix hater, let me know and I’ll remove you.
Please add other people you think need emotional outlets for their
frustration.
https://www.goodreads.com/en/book/show/174904.The_UNIX_Hater...
https://wiki.c2.com/?TheUnixHatersHandbook
>I'm a UnixLover, but I love this book because I thought it was hysterically funny. Many of the war stories are similar to experiences I've had myself, even if they're often flawed as a critique of Unix itself for one reason or another. But other UnixLovers I've loaned the book to found it annoying rather than funny, so YMMV.
>BTW the core group of contributors to this book were more Symbolics Lisp Machine fans than ITS or Windows fans. ITS had certain technical features superior to Unix, such as PCLSRing as mentioned in WorseIsBetter, but having used it a bit myself, I can't see that ITS was superior to Unix across the board. The Lisp Machine on the other hand, although I never used it, was by all accounts a very sophisticated environment for programmers. -- DougMerritt
https://news.ycombinator.com/item?id=13781815
https://news.ycombinator.com/item?id=19416485
>mtraven on March 18, 2019 | next [–]
>I founded the mailing list the book was based on. These days I say, Unix went from being the worst operating system available, to being the best operating system available, without getting appreciably better. (which may not be entirely accurate, but don't flame me).
>And still miss my Lisp Machine. It's not that Unix is really that bad, it's that it has a certain model of computer use which has crowded out the more ambitious visions which were still alive in the 70s and 80s.
>Much as the web (the Unix of hypertext) crowded out the more ambitious visions of what computational media for intellectual work could be (see the work of Doug Engelbart and Ted Nelson). That's a bigger tragedy IMO. Unix, eh, it's good enough, but the shittiness of the web makes humanity stupider than we should be, at a time when we can ill afford it.
https://medium.com/@donhopkins/the-x-windows-disaster-128d39...
I feel like that was one of the core assumptions and point of the article, but it didn’t have any explanation beyond “multiple programming languages.” Feels a bit flat to me.
> Why exactly?
Beside the defects well stated in the Unix Hater's Handbook, unix violate it's own principles since many years. Original unix idea was: desktops like Xerox SmallTalk workstations are too expensive and complex for most needs, so instead of a real revolution of an extraordinary outcome we decide to limit ourselves to most common needs in exchange of far less costs. No GUIs, no touchscreen, no videoconferencing and screen sharing [1] just a good enough CLI with a "user language" (shell scripts) for small potatoes automation and a bit of IPCs for more... Well... For more there is a "system language" (C) that's easy enough for most really complex task.
That was a success because no one really like revolutions and long terms goals especially if they demand big money while many like quick & done improvements at little price.
However in few years unix start to feel the need of something more than a CLI and some GUIs start to appear, unfortunately differently than original Xerox&co desktops those UIs were not "part of the system, fully integrated in it" but just hackish additions with so interoperability, just single apps who have at maximum cut&paste ability.
> Sure, but it also requires rewriting a lot of these things, introducing and fixing new bugs... It feels like the good ol' "let's rewrite this program" that quite frequently doesn't live up to the expectations
We need desktops again, witch means not just "endpoints" or "modern dumb terminals of modern mainframes named cloud", but desktop computing, since desktop development is essentially abandoned since many years and even back then was in a bad shape we need to restart from the classic desktops. LispM was ancient, hackish, but are still the best desktop we have had in human history so a good starting point. We have some kind of LispM OS/OE here: Emacs, still alive and kicking so there is something to work with, that's is. Emacs is already a WM (EXWM) have countless features and it's already "plugged" in modern bootloader OSes to have hw, driver and services. It just need to evolve.
[1] yes, you are reading correctly and no, I'm not wrong, I'm talking about the famous NLS "Mother of all the Demos" from 1968 https://youtu.be/yJDv-zdhzMY
> Why exactly?
Two reasons:
1 - systemd (which is moving linux towards becoming a systemd OS)
2 - developers and companies moving ever more towards web apps (which will eventually make the underlying OS irrelevant, as the browser becomes the OS) (incidentally, web assembly seems to herald the end of the open/transparent web too, as we're eventually all be running opaque binary blobs on our web browsers)
Or, a microservices OS.
Optane is the best performing SSD but the worst performing RAM you ever had. It is too expensive at any speed, even if Intel is losing money on it. HP memristors are vaporware.
LISP machines, Java machines, and similar architectures specialized for complex language runtimes are a notorious dead end. They just can’t keep up with performance-optimized RISC, pipelined, superscalar, SIMD, etc. architectures paired with compilers and runtimes that implement efficient abstractions (e.g. garbage collection, hotspot compilers) on top of those very fast primitives.
The vast productivity gains of Smalltalk and Lisp were because they discarded those abstractions and programmers were free for others.
Presumably OP posted this after noticing Phantom came up a few days ago. https://news.ycombinator.com/item?id=30807668
Elsewhere on this very site you'll find no ends of complaints about, say, Electron apps.
I dabble in microcontrollers for fun and there it's different. I am an AVR-8 fanatic and sometimes I think "this is so fast" and "2K of RAM is plenty" and "I can fit CRC-32 tables in 32k of flash because that's what counts as an 'operating system' for me"
Then there are the applications where it just doesn't have the power and I am so glad to have a box of RP2040's because in 2022 the most important attribute of a microcontroller is that it is available.
But Lisp at that time was already fast enough on standard RISC chips (MIPS, SPARC, ALPHA, POWER, ...). Later the 64bit RISC chips also provided enough memory space. SPARC also had some tricks for Lisp implementors.
Currently the assembler coded Ivory emulator is 80 times faster on Apple's M1 than the last Ivory hardware (the Ivory Microprocessor from Symbolics was released end 80s).
https://stackoverflow.com/questions/50966676/why-do-arm-chip...
>Why do ARM chips have an instruction with Javascript in the name (FJCVTZS)?
https://community.arm.com/arm-community-blogs/b/architecture...
Why would you want to do that other than hot patching a system that can't go down? Testing new changes requires more time than rebooting. If you just want to test simple changes, most debuggers can do that.
> Everything worked in a single address space, programs could talk to each other in ways operating systems of today couldn’t dream of.
And with a single address space you have win9x security.
> A modern UNIX system isn’t self-contained. I have 4 UNIX systems on my desk (Desktop, laptop, iPhone, iPad) I’m contentiously using the cloud (iCloud for photos, GitHub for text files, Dropbox for everything else) to sync files between these machines. The cloud is just a workaround for UNIX’s self-contained nature
This is just your use habbits. Nothing is stopping you from using NFS or SSHS. Someone who feels the need to use iCloud for whatever trivial convenience it provides is unlikely to benefit from a Lisp machine's ability to edit code on the live system.
> Then we add a gazillion programming languages, VMs, Containers, and a million other things, UNIX is a bloated mess of workaround for its own problems. We need a replacement, something that can be built for the modern world using technologies that are clean, secure, and extendable
The same thing will happen with any OS given enough time. Lisp is also not secure. It's prone to side channel and eval bugs.
> eliminate memory leaks and questions of type safety,
Lisp is not type safe.
It is type safe. While Lisp is not statically typed, its typing discipline is strong: operations performed on incompatible types signal recoverable errors.
Address space != protection boundaries. These are nearly orthogonal concerns. Where single address spaces might become less useful today is in dealing with Spectre vulnerabilities, though formalizing more explicit requirements about information domains (as in multilevel security, which is a well-established field of OS research) might help address those.
no it doesnt
> Lisp is not type safe.
yes it is
Typed Racket is, and that is why I love it
Then, I appreciated my M1 MacBook Pro running SBCL, LispWorks, Haskell, Clojure, and various Scheme languages - all with nice Emacs based dev setups. Life is really good on modern hardware.
> Life is really good on modern hardware.
Agreed: On modern CPUs.
More support for the additional hardware features like GPUs, media processing engines and the neural network engines (see the M1 Pro/Max/Ultra) would be welcome.
I can manage to context switch between different lisps fine but I do sometimes wonder in e.g. a slime+SBCL setup how much that context switching is costing me.
> Everything worked in a single address space, programs could talk to each other in ways operating systems of today couldn’t dream of.
No! Bad! We have enough problems securing software on separate VMs running on the same metal, single address spaces are completely out of the question until someone manages to build a feasible trusted compiler system.
> Then we add a gazillion programming languages, VMs, Containers, and a million other things, UNIX is a bloated mess of workaround for its own problems.
A lot of these problems could happen with a Lisp machine - you could have a billion different Lisps, for instance (although, to be fair, with better (i.e. non-Unix) OS design you wouldn't need containers).
> With lisp machines, we can cut out the complicated multi-language, multi library mess from the stack, eliminate memory leaks and questions of type safety, binary exploits, and millions of lines of sheer complexity that clog up modern computers.
This is partially true, but a lot of the complexity in modern software doesn't come from Unix, but just...bad design decisions. Webtech doesn't really care whether it's running on Windows or Unix, after all.
Also, high-level CPUs are a bad idea: http://yosefk.com/blog/the-high-level-cpu-challenge.html
I think the good in this post is along the lines of: text bad, typed IPC good, runtime-aware OS good, standardized VMs good, interactive systems (Lispy stuff, Jupyter) > batch-processing systems (Unix, C).
I know this is "uncouth" because it's commercial software, but Mathematica is one of the most interesting programs I've ever used. [edit] Might something like this be the future?
Oh man, wat?
I love lisp as much as the next guy.
But you absolutely can have library mess, memory leaks, and millions of lines of code using a Lisp. You arguably can have a "multi-language" mess, too, because Lisp gives you wonderful tools to create DSLs; I'd say creating a language that fits your needs, and then using it, is the right way to use Lisp.
I use Emacs daily, and see how an all-Lisp environment can make for a good, productive interactive experience. More efforts in this area would be quite welcome, but this is a shell, not a kernel.
I still suppose that systems software, and especially the key parts of an OS, need a language more like Rust than like Lisp, with a good affinity to raw hardware, and a ton of static guarantees.
The "lisp epiphany" is real. Either you "get it" or "you don't".
I've been writing lisp programs for 50 years. I've been paid to program in 60 different languages but nothing compares with lisp.
There is an intellectual "distance" between a problem and its machine solution. I call this the "impedence problem". Lisp lets you think at the most abstract and write to the most specific.
Writing changed the world. But if you give most people a blank piece of paper they don't know what to do with such freedom. Lisp is the "blank piece of paper" of programming languages. Everything, literally everything, comes from you.
I loved my Symbolics machine. It was the closest expression of a "thinking platform" I've ever used. IDEs are horrible for thinking, ever interrupting at every keystroke.
Lisp isn't "popular" because it provides a "thinking platform" you can shape to your thoughts.
Lisp will never be popular. The reason should be obvious.
My claim is that Lisp is perfect for thinking about a new idea or a new approach.
For example, I implemented a program that merged Expert Systems and Knowledge Representation into a single system (KROPS) that allowed a domain expert to express their knowledge as rules or as facts. Anything the system learned by either method could be expressed in either representation. Thus, the two representations were "unified".
Another effort involved Human-Robot Cooperation to change a car tire (TIRES). The system could interact with the human through pseudo-natural language, learn rules dynamically, and expand its knowledge base of the current situation in real time. So the system self-modifies and learns through human interaction on the task.
Both of these systems required self-modifying code which is rather more difficult to do in other languages. In Lisp this is trivial.
As for commercial sales witness:
Axiom, a 1.2 million line Computer Algebra program written in Common Lisp, was sold commercially by the Numerical Algorithms Group.
YESOPS, an IBM Expert System program implemented in Common Lisp, was sold commercially.
I am glad to hear that.
Why be frustrated? For example I'm using the latest Apple Silicon laptop and there are a dozen Lisp (and related) systems already ported to it. Years ago it took a lot longer to move to a new platform, especially for open source software implementations with native code compilers.
Happy times.
I've been learning/using CL, on the side, for about a year, in fits and starts, and I'm also picking up a lot of it's history and evolution along the way.
I find the history of the thing is as fascinating as the language/tools.
There's so much written on this but it's hard to aggregate and put together. Links lead to other links that lead to other links and oftentimes a lot of them are dead.
So I'm glad that there are books like this that can preserve some pieces of the story.
The name of the game has always been to avoid directly insulting the bad programmers by making fun of the languages they use. Bad programmers have clustered in several languages over the course of my long career. Mostly they are the easiest most expressive languages, which would superficially seem to be an advantage, however they make it easy to express amazingly bad ideas. The harder to express languages require more work to express a bad idea thus somewhat filtering them out of that language's pool.
People whom don't understand the game think the game will be won if the bad programmers would have access to better languages for the first time in history. Despite the idea being decades old its always presented as a new idea.
There's an authoritarian streak where if only we could remove access to the inferior languages then they'd have to use the better languages and we'd have better code. However you can't force people to not use inferior tools and you can't force them to learn to use better tools.
The graph of easy to write and code goodness is interesting and nonlinear. You can express very complicated ideas in lisp easier than in vb6 or perl or interpreted basic or spaghetti fortran. However, you can express very bad ideas easier in a "bad" language, so it accumulates interesting authors. There is always a crossover point where an intermediate complexity idea is equally hard to express in a simple language or a complex language. Frankly most of IT needs are and always will be below that point. So its counterproductive to demand difficult language for simple tasks, everyone laughs at "Enterprise Java Hello World" that is 100K lines of enterprise patterns.
Expressing ideas in computer languages is much like expressing ideas in everyday language. Some esoteric philosophy texts require a VERY large precise complicated hard to use and hard to learn vocabulary. Road signs do not. For in between jobs, trying to use a minimum number of language vocabulary words and lexical complexity would be wise. It would be a fools errand to try to cut half the vocab words from street signs to "make driving safer", or an equally bad idea to force all road signs to be expressed as Shakespearean sonnets. The real world counterpart of "Enterprise Java Hello World" would be forcing the "No U Turn" street sign to be in the form of a Shakespearean sonnet.
The difficulty of tasks is usually under a power law, so its nice that we have lisp, but usually a bad idea to use lisp.
I imagine that a lot of this can be replaced with libraries for a general-purpose language like Python. In the case of Awk, BC and DC, Python's standard library does everything these do without their strange quirks. Gnuplot can be replaced with umpteen plotting libraries like Matplotlib. Shell-scripting can be done in a Python dialect like Xonsh. I don't know of a Python alternative to Make, but Make is a fairly perverse and ad-hoc language that looks ripe for being replaced by a library.
I don't think a new operating system (however you define that) is necessary. I think you just swap out a lot of the crazy DSLs with one consistent general-purpose language. People will switch over when they want to accomplish things more easily (like me!). This is especially likely if you're not a SWE but you want to do file system automation anyway. The DSL-heavy approach is too Byzantine for such people (like me!). And it's mostly possible today.
It can all be incrementally replaced over time, without starting over, just like how systemd and pipewire didn't need to totally start over.
Make will probably need something more than just a library though, because it's gotta stay declarative. But the fact that make exists at all is kind of an issue. I think build and package management should be done in the language itself like non-C languages do.
The other use of make is almost as a pseudo UI, just a standard place to put a list of actions you can do.
Something like Ansible could replace that, in theory, or we could have some new "project control center" file with menus and inputs and settings.
I think Gnu Guile can integrated with Make
While the Plan9 mouse chording is cool (and could be kept), I would have everything also accessible via keyboard commands. 3 button mouse chording on a laptop trackpad is not fun.
Which is annoying, because both of these approaches produce some really attractive results, so I’d very much like to learn about any attempts to reconcile them.
A unikernel that throws out the legacy of Unix and starts fresh to build a library operating system, it's exactly what OP describes:
> With lisp machines, we can cut out the complicated multi-language, multi library mess from the stack, eliminate memory leaks and questions of type safety, binary exploits, and millions of lines of sheer complexity that clog up modern computers.
Even better, Mirage is programmed in OCaml, which is basically a statically-typed facade over Lisp (or Scheme). That's the modern Lisp machine of today. It even takes care of security nightmares like this:
> Everything worked in a single address space, programs could talk to each other in ways operating systems of today couldn’t dream of.
Because in the Mirage model is program is a separate OS image and they can communicate only over defined service interfaces.
We should hope to see a flourishing of new and diverse systems again, since now all one need to do to interact with everyone else is merely (and I know it's a slog) implement tried and true standards.
They supported things like TCP/IP, UDP, SMTP, X11, RPC, NFS, DNS, HTTP, ... The machines had C compilers, too.
So, yeah, standards and protocols are there. And it would take a s*t-ton of work to implement them in a bespoke environment. But they are not "difficult" in the classic sense. If we had a completely different type of economy it might even be possible!
This suggests it's too much work for now if you're just curious, but it'd be great to be wrong.
> These machines used specialized hardware and microcode to optimize for the lisp environments (Because of microcode you could run UNIX and the Lisp OS at the same time).
This is a dead end in the history of CPU design.
Processors are all vaguely similar these days. Your CPU is built around ALUs which typically take one or two inputs, produce one output. Around that, you build some logic for shuttling these inputs and outputs to and from registers, or in some cases, to and from memory.
The core here, the ALUs, have gotten more and more sophisticated over the years, but the wiring on the outside has remained fairly modest relative to its excesses back in the day. I'd say that the lesson here is simple: rather than add complicated operations to make your high-level language faster, do the complicated stuff in software... which gives you a lot more flexibility, and the combined hardware-software stack ends up being faster and cheaper anyway.
> With lisp machines, we can cut out the complicated multi-language, multi library mess from the stack, eliminate memory leaks and questions of type safety, binary exploits, and millions of lines of sheer complexity that clog up modern computers.
I can understand where this notion is coming from... but practically speaking, switching to Lisp doesn't eliminate memory leaks or questions of type safety or binary exploits. Even with an idealized version of Lisp, I don't think these problems could possibly go away. Neither garbage collection nor systems like Rust really "solve" memory leaks, they just provide strategies to make memory leaks less common.
The same thing applies to type safety. You'd have to define "type safety" very narrowly to say that Lisp solves all type safety problems. Again, I can understand where the author comes from--it's kind of an intuitive notion of type safety, that you don't end up operating on an incorrectly typed pointer, or something like that. But the notion of type safety is much more broad than that these days.
And the C/Unix strategy is actually pretty good, too, when it works--contain memory leaks within a process, then terminate the process.
The issue I see with the idea of lisp-all-the-way-down is that most people don’t want to write filesystems and device drivers. I know I don’t. I mean I find them fascinating, but I usually have a specific app I want to write. I don’t want to write device drivers on my way there.
So this is basically expanding Emacs to actually be the operating system. There's definitely some allure to it. Awhile back, I was hacking on the Lem editor for Common Lisp, using the Lem editor itself. It was great until I made a mistake in some code that restricted my ability to edit the code, sort of like performing brain surgery on my self and snipping a nerve that controlled my arms. It was amazing to have that immediate feedback loop, but in a world that is striving to find new ways to minimize human error, I'm just not sure it'd hold up.
(I've been pondering this problem recently for something I'm working on an introducing it from the ground up is bending my brain hard enough, others may be better at it though ;)
This seems like the kind of goal that's only palatable to a very few people nowadays. Specifically, the people who want to use that language, toolchain, and libraries, and nothing else.
These days, I don't think that that's ever going to allow for enough of a community to support more than a relatively self-contained hobbyist scene. Which there's absolutely nothing wrong with that; personally I wish there were more compelling tinkering-oriented platforms; I'm a little meh on Unix too. But the article seems to be advocating rather loftier ambitions.
And I mean it, Emacs is as close to a Lisp Machine as you get in today's world.
The evil package.
Here's a demonstration of Symbolics Open Genera (TM) 2.0, demonstrated running in a virtual machine. It is noted by the author of the video in the first minute or so that even in emulation, it is much faster than the original machines. - https://www.youtube.com/watch?v=o4-YnLpLgtk
Oberon also had a similar attribute, in that they kept the names of functions to operate on objects visible.
The same was true of the Canon Cat, Hypercard, Ted Nelson's Xanadu project, Smalltalk, and a number of other early computing systems.
The main feature common to all of these systems is that they all preserve context. In Genera, Oberon, Canon Cat, Hypercard, and SmallTalk the source was always available. (as far as I know). In Xanadu, the main functionality of the web was present, but it wouldn't allow the broken links (and lost context) that now plague the web.
I think a future platform could take code in a number of languages, compile it to an abstract syntax tree, but preserve the context required to recreate the source. In fact, it's reasonable that you could import a routine in a language you aren't familiar with, and reverse the compilation to get it expressed in an equivalent (but less elegant) form in your language of choice, along with the original comments.
There's nothing stopping an open source project from taking elements of these existing systems and moving forward from that basis. It might be profitable to include ideas such as Coloring of the Source text to label intent, such as in ColorForth.
Also, consider "Literate Programming" - Literate programs are written as an uninterrupted exposition of logic in an ordinary human language, much like the text of an essay, in which macros are included to hide abstractions and traditional source code.
You could also add the ability to store graphics and other data along with the source code.
Of course, if you are required to run code you didn't write and don't trust, your operating system must provide means to run it only against the files or folders you wish to let it operate on. The principle of least privilege needs to be supported at a fundamental level. This is one of the big shortcomings of the Unix model.
Sorry it was a bit of a ramble, but this seemed to be a call for ideas, so I ran with it.
PS: In the past, getting your vision of Computing required building a machine, and then getting it manufactured. Now it just requires that you make it work in a VM, Raspberry Pi, or web browser window.
It is MUCH easier to try out and/or create alterative systems now that it has ever been in the past.
Now if Windows team would get over their COM worship, it would be even better.
If you arrive at "we can't do that because everyone wants different things", you're in this weird place where you can't build one system to suit everybody so as a fix for that you will ... make one dev system to suit everybody? Why is that going to work out any better? If people are going to have to build the system they want anyway, why not skip all the middlemen and leave people to build the system they want from scratch their own way?
"Well we can't do that, having to build everything from scratch is hard, people don't want that. And we don't know what people want (and can't be bothered to ask) and won't try to build it for them, but we can give everyone Our Blessed System and Our Blessed Tooling which we presume they will want to use, while we abandon them to have to build what they actually want using it". It's patriarchal and preachy and still unhelpful. The kind of person willing to put up with your system and language and its edge-cases and limitations instead of making their own, is quite likely the same person who would have put up with your spreadsheet and its edge-cases and limitations if only you'd built it.
It's the "all progress depends on the unreasonable person" meme; you need the person who can't tolerate a system which isn't perfect and demands to be able to tune it to their perfection, but is simultaneously fine with a customizable-system built by someone else for someone else's ideas. Then you say "ah but they can rebuild as much of the system as they want!" they're now having to build it themselves from scratch but based on what you built, which is even more work for them, not less.
And the whole thing says little about working with other people/organizations; everyone building their own thing isn't great for that. One could argue that Microsoft did a lot of good for the world by strongarming companies into .doc and .xls formats for interchange, in a similar way that HTTP/HTML did (but more tyrannically). Microsoft-Word-documents-over-email should probably go down in history as one of the big enablers of distributed human organization, like the telephone. Moreso than SMTP alone or TCP/IP alone which enabled computers to connect, not people. More than HTTP/HTML which ordinary people can't use without setting up a webserver and publishing first.
> "I think a future platform could take code in a number of languages, compile it to an abstract syntax tree, but preserve the context required to recreate the source. In fact, it's reasonable that you could import a routine in a language you aren't familiar with, and reverse the compilation to get it expressed in an equivalent (but less elegant) form in your language of choice, along with the original comments."
Would that even work? What about "code is primarily for people to read"; loss of indentation, of alignment, of variable names if you're changing language with different variable naming rules, loss of separate files/folder/module-context of how it was built in the first place, and loss of idiomatic patterns from changing one language to another, or to languages with different scoping behaviour. Look at the image in The Article for the square root code[1], can you decompile the syntax tree for a language which doesn't have a complex number type? What about one which doesn't have a short-float for the result, or has different floating point precision assumptions? What does "F and I2 need to be regular-heap-consed to avoid the extra-pdl lossage" mean for JavaScript or Python or Clojure? What's the magic 0.4826004 for, without any comments next to it?
And that square root function alone is a screenful of code, the very idea of being able to rebuild the system to your desires falls out the window if it's going to take you a lifetime to do that, and if not then we're back to my original paragraph where you're building a massive programmable system so someone can tweak one or two tiny things occasionally, which seems hella overkill.
> "Of course, if you are required to run code you didn't write and don't trust, your operating system must provide means to run it only against the files or folders you wish to let it operate on. The principle of least privilege needs to be supported at a fundamental level. This is one of the big shortcomings of the Unix model."
The XKCD comment that malicious code can ransomware all my documents and upload them to the cloud, but at least it can't add a new printer. One of the big shortcomings of the principle of least privilege is the effort of carefully and precisely describing the privileges needed - see the complexity and quantity of SELinux rules for a whole system. Even then you get immediately to the point where "my editor can edit all my text files" and that's now a large blast radius.
[1] https://cdn.substack.com/image/fetch/w_1456,c_limit,f_auto,q...
SELinux was a terrible thing to inflict upon the world. Capabilities are more like cash in your wallet.... you pick what you want to use in a transaction, and hand it over... never worrying that the rest of your money would somehow get siphoned off later.
Static rules aren't what capabilities are about. Run this program, oh.. it wants a file? Here... it can have this. The default access would be nothing (or in a more modern context, it's config file) The program wouldn't directly access files, but would use dialog boxes to do I/O, or you could drop them in via batch, etc.
Because what you want is not set in stone for all time. Needs and desires change, and often what you think you want is different from what you actually want, but discovering that can take time.
I spend so much time trying to work out things like "when I click this button in the UI, what does that actually end up doing in the backend?" I wish I could right-click on a UI element and it would take me directly to the backend code which actually implements it. And a system like that should be much quicker to change – it should be much quicker for a new developer to get up to speed with it and start implementing things, rather than spending days (even weeks) trying to understand how it all fits together.
https://news.ycombinator.com/item?id=7878679
http://lispm.de/symbolics-lisp-machine-ergonomics
https://news.ycombinator.com/item?id=7879364
eudox on June 11, 2014
Related: A huge collections of images showing Symbolics UI and the software written for it: http://lispm.de/symbolics-ui-examples/symbolics-ui-examples....
agumonkey on June 11, 2014
Nice, but I wouldn't confuse static images with the underlying semantic graph of live objects that's not visible in pictures.
DonHopkins on June 14, 2014
Precisely! When Lisp Machine programmer look at a screen dump, they see a lot more going on behind the scenes than meets the eye.
I'll attempt to explain the deep implications of what the article said about "Everything on the screen is an object, mouse-sensitive and reusable":
There's a legendary story about Gyro hacking away on a Lisp Machine, when he accidentally trashed the function cell of an important primitive like AREF (or something like that -- I can't remember the details -- do you, Scott? Or does Devon just make this stuff up? ;), and that totally crashed the operating system.
It dumped him into a "cold load stream" where he could poke around at the memory image, so he clamored around the display list, a graph of live objects (currently in suspended animation) behind the windows on the screen, and found an instance where the original value of the function pointer had been printed out in hex (which of course was a numeric object that let you click up a menu to change its presentation, etc).
He grabbed the value of the function pointer out of that numeric object, poked it back into the function cell where it belonged, pressed the "Please proceed, Governor" button, and was immediately back up and running where he left off before the crash, like nothing had ever happened!
Here's another example of someone pulling themselves back up by their bootstraps without actually cold rebooting, thanks to the real time help of the networked Lisp Machine user community:
ftp://ftp.ai.sri.com/pub/mailing-lists/slug/900531/msg00339.html
Also eudox posted this link:
Related: A huge collections of images showing Symbolics UI and the software written for it:
http://lispm.de/symbolics-ui-examples/symbolics-ui-examples....
Dynamic typing is great for prototyping, early development, and for small teams. If your definition of success is greater than that I would choose differently, or port at a good time early on.
And if someone were to ask me to join a company using lisp, I would hope that it's Clojure, though arguably not a Lisp, is better it that it will vary less between usages. This gives it a better chance of a growing ecosystem.
Also not sure LISP machines solved security as well as modern-day Linux does. I think it was just less of a concern back then, because you knew most of the people who were on the network.
eh, if it's gui is anything like clim which was based off of it, no thank you big time
Linux is slowly becoming a standardized, integrated platform. I don't see why it can't be evolved to have all the main advantages of a LISP machine.
I also don't see how that solves dependency management. No matter what, if you build against something and it changes, stuff breaks. That's the main issue with Linux.
It also doesn't solve microservices being kinda hard and needing manual configuration specifically for the setup, rather than the one size fits all style of monolithic desktop software. That's an application architecture challenge.
Nor does it solve cross-platform.
Linux does have problems and could learn from LISP machines(Although I'd rather we have TypeScript machines or Python or something, LISP is pretty far from what I want a language to be, and is meant for creativity and expressiveness rather than Ada-like safety and boring hacker-repelling Java-like standardization).
But a lot of issues go away if you pretend everything other than Debian and Red Hat don't exist.
I remember seeing GNU ls code, 30% argparse, 30% formatting.., all of this seems brittle and redundant.
Bazaar is fine to allow freeform innovative evolution but it's also too messy.
(Not always great though; I don't know VMS shell but PowerShell cmdlets can still have similar tools taking different parameter names for the same concept, or not working implementing what to do with common parameters, or taking DSLs in strings like WMI Query Language, or taking rather opaque and unique hashtable blobs).
Most people using Linux also have DBus and Pulse/Pipewire going, and a ton of other stuff.
Network devices are technically files, but in practice they are names. I have no idea what them being a file even means at the byte level. They're just names that you deal with through NetworkManager's DBus API, if you're not targeting any of the minimal or embedded distros.
C was created to make UNIX, originally written in straight Assembly, portable. A process finalized by UNIX V6 release.
UNIX philosophy grew out of Bell Labs into all universities and business that took those source tapes, and took it from there.
Hardly possible to be selective of what UNIX is all about.
It's evolved so much, and the remaining unixy bits get more and more supressed and hidden.
BSD still has more of it, but Linux seems to be moving on and is very far from the days of everyone piecing together software from small parts.
Most projects using the Linux kernel nowadays care more about having a free beer kernel than anything UNIX related.
BSD has always been a proper UNIX, hence AT&T lawsuit when they were finally allowed to monetize on UNIX.