Hell Is Other REPLs
hyperthings.garden
hyperthings.garden
The thing about "private understandings" is that they're just another way of saying "we expect ceaseless, flawless vigilance" (as one writer put it), not to mention "flawless communication and training."
Languages which impose weird restrictions tend to do so because it allows them to offer (nearly) ironclad guarantees about something else.
There are certain programming idioms that work brilliantly in Haskell. Those same idioms would be utterly miserable in the presence of unrestricted mutation.
Or to take the author's other example, Rust forbids shared mutable state, and it keeps careful track of ownership. But I can freely use native CPU threads without worrying about anything worse than a deadlock, and I can bang directly on raw bytes without worrying about anything worse than a controlled runtime failure. And this remains true even if team communication occasionally fails or if someone makes a mistake.
Sometimes I want to rule out entire classes of potentially dangerous mistakes, and not just have a "private understanding" that nobody will ever make certain mistakes.
As always, it's a matter of using the right tool for the job. If you need to write a high-performance, heavily-threaded network server that parses malicious binary data, Rust is a great tool because of those restrictions. If you need to do highly exploratory programming and invent new idioms to talk about your problem domain, Common Lisp is awesome. And if you need to build libraries where everything has a rigid mathematical structure, Haskell is a great tool.
In my experience, Commony Lisp is a deeply opinionated language, and its most dramatic opinion is that "your code and your data should have identical representations and structures." And for the right problem, that restriction is extremely powerful.
Nice in theory. In practice a manager will ask you to build A, then later they will ask you to add B and C, and these components should interact seamlessly of course. If you chose your language based on A, then you might get stuck on B and C.
Pretty much every app, driver, and framework I've worked on across the web, desktop, and mobile has been mostly glue holding different components together. Choosing a different language would require a nearly-complete rewrite in every case except for a tiny section of independent interesting logic (and of course, if you arbitrarily picked some meme language for your "business logic" you'd have to write more glue code to attach it to the rest of your software..)
Anywhere you're e.g. dealing with an eventually consistent database... that is not business logic, that is all glue code that exploded due to scaling concerns.
Was this mad? Yes. Was it quicker to market than a rewrite? Yes.
I've done a few total rewrites now, and the only one I am sure was a good idea was for this reason (DPX, the Java re-implementation of Garlik's core software before it was purchased by Experian Consumer Services many years ago).
- "Our web apps get written in Typescript."
- "Our data science gets written in Python."
- "Our inner loops get written as Rust CLI tools that contain no business logic." (Or whatever. It depends on the problem domain.)
This works because TypeScript, Python and Rust are all very good at certain tasks, but they're each also fine general-purpose languages with lots of third party libraries. If you do somehow wind up writing a webserver in Python, you'll be fine. Of course, something like Haskell is a deeper commitment with more tradeoffs, and it's the wrong answer for most organizations. (But if I were mathematically modeling the pricing of complex derivative contracts, Haskell would make the list.)
But this is why good senior people with taste are so useful, and why competent technical management is worth its weight in gold. Sometimes you want a toolbox with 2 or 3 well-chosen, versatile tools. And sometimes you get organizations that try to pound in nails with a screwdriver.
Our web apps get written in Perl (later added: with jQuery for FE interactivity)
Our data science gets written in Oracle stored procedures
Our core business logic is written in Oracle stored procedures!
Fat clients are written in Smalltalk
These were arguably even sort of well chosen at the time. (I won't say which company this was as that would identify me and I changed some details but this is a real life example)
For example I've seen:
- COM and ASP.Net to Java and JSF (general web apps)
- Forté 4GL (look it up, it was very interesting) to J2EE (general enterprise software)
- MIDP Java to React Native, following mobile phone evolution (mobile app front ends of general enterprise software)
- HTML frame messes to Java applets to JavaScript interactivity (high profile public web site)
- ColdFusion, or something of the sort, to Sharepoint (another high profile public web site)
- SharePoint to a BPEL workflow engine (important workflow-oriented application)Not using COM for native programming on Windows is being stuck with Windows XP.
How so? You can do whatever paradgim you want in CL. There is no policing involved. Not in the sense of Haskell or Rust at least. CL will let you modify globals or share data willy nilly just fine.
Mixing functions and values in lists is not really a restriction either, it opens up for all kinds of shenanigans.
And if the abstraction bedrock is high, then problem domains below that bedrock can't be approached with the language.
Various Common Lisp implementations like ABCL (Common Lisp on the JVM), ECL, CLASP, mocl, ... don't support saving and starting images.
> And tree-shaking is less effective because the language is so dynamic it's hard to guarantee some code won't ever be called.
That depends on the delivery system. For example in LispWorks I can manually remove a lot of functionality:
http://www.lispworks.com/documentation/lw71/DV/html/delivery...
[0]: https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
Another one was CLICC : https://github.com/hoelzl/Clicc which has a language definition for CL0 : https://github.com/hoelzl/Clicc/tree/master/doc
There are or were a bunch of such small delivery oriented implementations: Oracle bought one years ago, Gensym has one, mocl is derived from CLICC, ...
L was actually a subset of Common Lisp. It would run pretty nicely on a machine with 1 MB total RAM and a 16 MHz CPU. All the basics were there: lambdas and macros and symbols and modules, and they all worked how they did in Common Lisp. (Or they were close enough that you could write and test in Common Lisp and then cross-compile.) There was a REPL that could run most code, and an ahead-of-time compiler.
It was a fantastically nice environment for embedded systems too small to run something like Alpine Linux. You could compile a program, load it onto actual hardware, and then tweak things interactively using the REPL. And you could invent all kinds of macro DSLs.
Of course, all this becomes moot once your system is large enough to support Alpine Linux and an SSH server. If you can run Linux, then you can use Lisp or Python or PLT Scheme or Rust or anything else you want, and you have half a million open source libraries available.
Still, L is proof that Lisp is a great embedded language for anything with a MB of RAM. You can have a pleasant development environment and tons of flexibility, with an excellent REPL.
The idea of CLICC was to compile programs to small static C programs. Really small and with little or no dynamic features of Lisp (code loading, runtime compilation, eval, redefinition, ...). It did not have all the bells and whistles of ECL (virtual machine, runtime code loading, debugging, full dynamic Common Lisp, ...).
- You should be able to implement Lisp-in-Lisp within a few pages of code. This a profound and remarkable design constraint. (See http://languagelog.ldc.upenn.edu/myl/llog/jmc.pdf (PDF) for an introduction.)
- Syntax would only hide the fact that code is a simple data structure that can be manipulated like any other data structure.
Just like Rust's strong opinions buy you stress-free threading and bit-banging, Lisp's opinions ensure that it's an almost ideal environment for inventing domain-specific language extensions.
But Rust is poorly suited to object graphs where everything mutates everything else, and Common Lisp is poorly suited to proving that a large program obeys certain strict rules.
I love opinionated tools, but I try to choose ones with the "right" opinions for the task at hand.
This is particularly relevant for certain algorithm, such as atomic initialization, that get significantly more complex with in-line ownership tracking.
http://db48x.net/reposurgeon/rust-port-docs/reposurgeon/path...
Also, because of Rust’s wonderful type system it will be fairly straight forward to replace the string keys in this code with interned strings instead. (The amount of strings we use for paths in a large repository is pretty astounding.) That might now be almost possible in Go, if I learn the new generics.
Certainly not everything is a good fit for Rust’s ownership model, but honestly most code benefits from it at least a bit, and steadily more and more is being figured out about how to mesh other models into it without too much pain. It’s very liberating, protecting from various mistakes that are too easy in other languages (and that’s specifically where I miss it).
Ownership modelling can require more effort initially if the problem doesn’t match Rust’s capabilities very well (or put another way, if the problem doesn’t match how hardware works without garbage collection), but subsequently it’s liberating to not need to worry about various problems that are ubiquitous in other languages.
I reckon it similar to the debate of static versus dynamic typing, though more subtle. For toys, dynamic typing is adequate and static typing a burden, but as you scale a system up, dynamic typing makes life harder and requires more discipline to avoid ossification or drowning from technical debt. It’s taken time, but there has been a significant resurgence in static typing as people slowly come to realise its value through bitter experience in dynamic typing. Like static typing, a strict ownership model requires more effort initially, but makes life easier down the path, and I expect that at least the basic concepts of it will be picked up in more languages new and old over time (this has started already, actually, with Swift and D springing to mind), though it’s probably a harder concept to beneficially retrofit to a language than static typing.
You may want to mutate and forget about ownership, but Rust doesn’t let you do this, and it’s for your own good ;-). In time you get more of a feeling of why it is the way it is and learn to appreciate it.
As I'm sure you know, a Lisper would probably design a domain specific language that was well suited for such proofs. One of the projects I'd like to get around to one day is a Lisp hosted general purpose DSL[1] based on predicate transformer semantics[2] that combines writing specification and program text into a single interactive experience. Think something along the lines of smartparens[3] strict mode, where the editor doesn't allow you to make an edit that will violate the count("(") == count(")") invariant and you modify the tree structure with operations like slurp and barf, but a bit more general.
[1] Haha, I know.
[2] https://en.wikipedia.org/wiki/Predicate_transformer_semantic...
Even if you do need to model an arbitrary graph (for example), though a traditional programming language would make it a lot easier than in Rust, I still wouldn't necessarily let that dictate the choice of programming language. Most likely your program will end up doing a lot more than just banging on the graph and those parts may benefit from a more disciplined programming language.
Not recognizing this (like in the article) is a clear sign of a single-paradigm developer that never really understood anything else but is too arrogant (insecure maybe?) to assume he doesn't understand something. So, the problem must be with everybody else.
Anyway, a funny red-flag is that reaction to Haskell IO. People that never really learned Haskell tend to complain about IO with a deeper complaint of "why to I have to keep using monads?", while after people learn it they tend to complain about the lack of monad composition, with a deeper complaint of "why can't I use monads for everything?"
But for solving several of them, you need to compose your monads, and there is no good general way to do that. There are context dependent ways to compose some of them, but you are always thinking about their compatibility and they often require that you specialize your code.
It's (provably, I think) impossible for fully general monads, but for traversable-but-otherwise-general monads you can do[0]:
newtype (%) (f1::k1->Type) (f2::k2->k1) (a::k2) = Comp { unComp :: (f1 (f2 a)) }
instance (Monad f1,Monad f2,Traversable f2) => Monad (f1 % f2) where
join (Comp a) = Comp $ a >>= (map join . mapA (unComp ))
(Comp a) >>= k = Comp $ a >>= (map join . mapA (unComp . k))
0: copied from my own code, so it might need other adjustment than s/map/fmap/ and s/mapA/traverse/.
If dealing with Untrusted File Formats perhaps you should use a tool purpose-built for Wrangling them Safely, WUFFS.
You won't find a Hello, World example for WUFFS because Hello World prints out text to your console, which is exactly the sort of nefarious stuff bad guys might try to do and so WUFFS doesn't even provide any mechanism you could use to do that even if you wanted to. But it does Wrangle Untrusted File Formats Safely, shrinking your remaining problem space.
For example WUFFS would be appropriate for taking files your users claim are JPEG "photographs" they uploaded for their "profile picture" and turning each one into either a 64x64 pixel RGB array or an error without any risk that they seize control of your profile picture program and do goodness knows what else instead.
Although Rust's memory safety means you can achieve confidence a "photograph" doesn't corrupt memory it doesn't require the rigour WUFFS brings to file parsing, so a Rust program could end up confused about unforeseen state while parsing the file. For example in Rust you can write a function that might mistakenly overflow a 32-bit integer and, in production it will just silently wrap. In WUFFS that function won't compile until you either decide explicitly what should happen (e.g. wrapping, saturation) for each overflow, or you trap all the cases where it could overflow as an error. This is very annoying of course, but we're parsing Untrusted File Formats and if we leave anything to chance that will be exploited.
I'm pretty confident that I can parse untrusted binary data in Rust with nothing worse than a denial of service. (And I have over a billion 'cargo fuzz' iterations to prove it.)
But WUFFS is even more opinionated and strict than Rust, and so it can offer even stronger guarantees. Admittedly, it looks pretty obscure and the documentation is a little light, but it's a great idea.
I am really just done with CVEs, or at least the sort of CVEs that appear in C programs. We know how to prevent so many classes of security holes completely, allowing us to focus on the harder challenges.
You can do this in Rust, up to a point. There's a lint to ban dangerous arithmetic: https://rust-lang.github.io/rust-clippy/master/index.html#in... . You can then use the {saturating,wrapping,checked,overflowing}_{add,div,mul,abs,...}() methods to decide exactly what should happen on overflow.
But WUFFS seems a lot nicer. Judging by the README it first tries to determine whether overflow is actually possible, while Clippy will happily forbid you from running "1 + 1".
Of course, using a formal methodology like B, ceaseless, flawless vigilance is mandatory.
[0] https://www.cambridge.org/gb/academic/subjects/computer-scie...
> In my experience, Commony Lisp is a deeply opinionated language, and its most dramatic opinion is that "your code and your data should have identical representations and structures." And for the right problem, that restriction is extremely powerful.
Code in common lisp is almost entirely in a tree structure via nested linked lists. However, when programming in Common Lisp, I don't use this specific data structure very often for things that aren't code. Furthermore many other Common Lisp programmers write software this way as well, which is a bit of a counterexample to the assertion I was replying to.
[edit]
I think an opinion common lisp does have is that code should be easily manipulable as data. As much as scheme programmers like to call CL macros "unhygienic" it's far easier to write a correct general-purpose CL macro than a correct macro in other languages.
Yes, but oftentimes this is more like:
"If we tie you to this bed, you will never have a car accident! You'll also never get lost! This also makes it easier to stay healthy as we'll be feeding you the best food!"
Sure, but I'll also miss all kind of activities I could be doing outside on my own. I'll also get bed sores. And I often want a steak or ice cream, even if it's not ideal, but you insist bringing me this organic gourmet vegan crap.
No one sets out to create programming languages that tie people to their beds. If people add restrictions, it's not to make users miserable.
It's fine if you like being unrestricted, I like that too.
However, when you see people going out of their ways to add barriers to their own work, you should assume they reap a benefit that you're not fully appreciating, not that they hate fun and want to outlaw ice cream.
People can still end up designing such languages, even if they didn't set out with that specific goal.
(Kind of like how you can bring war and division, even if your goal is to bring some religious peace on Earth or "the revolution").
>If people add restrictions, it's not to make users miserable.
That's already explicit in my comment though. They didn't tie the person to his bed to make him miserable but to spare them from car accidents, to help them never get lost, and other such niceties!
The misery is a by-product, not a design goal.
They mean well. Fine. I still don't have to like it, and I don't have to accept it.
For example, as have been shown for example countless times, C programmers are unable to properly free allocated memory so solutions were made (GC, or rust’s ownerships). It is egoistical to think
The point is, you don't have enough information to choose for me. You can't do it. Even if you can absolutely, 100% correctly decide what the "right thing" is in any given situation, you don't know all the situations.
(And if you'd say "the right thing is to not use Pascal in that situation", I'd agree with you. But as a language designer, your language will get used in places where even you may think it's not a great fit.)
Yeah, it might suck that you can get a ticket for treating a red-light like a stop-sign at 6am in the middle of no-where. Though this is me speaking from a "likes rust" perspective.
Rust has unsafe, Haskell has unsafePerformIO. You might not like how verbose some of the stuff is but all these languages give you a foot gun if you really want it.
A well known article, written in bad faith, is "Why Pascal is Not My Favorite Programming Language".
http://www.cs.virginia.edu/~evans/cs655/readings/bwk-on-pasc...
Why bad faith? Most of the critic he makes against Pascal was sorted out by Pascal dialects.
Which one can consider doesn't count as dialects aren't the main language, yet by 1981, C was mostly available in K&R dialects outside UNIX, like Small C.
And besides those Pascal dialects, Modula-2 standard was released in 1978, while Mesa was powering XDE and Bravo at Xerox.
That reminds me of Yes, We Have Noticed The Skulls [0].
[0] https://slatestarcodex.com/2017/04/07/yes-we-have-noticed-th...
Arthur Laffer is an economist. Laffer has helped provide advice to numerous Republicans and Republican projects which of course didn't achieve their stated goals (e.g. Trump's massive tax cuts for the wealthy which Laffer claimed would grow the US economy by 6%).
I'm confident Laffer has Noticed The Skulls, but why should he care? Advice that destroys the US economy but further enriches those who already have more than enough worked out great for Laffer.
This is not about some "technical" impossibility - language still have an escape hatch like unsafe and "non-idiomatic" ways of coding.
But to focus on the technical and ignore programmer experience ("feelz") in that area, is like equating Idris and Forth because "they're all turing complete and it all ends up as binary code anyway".
Whereas working with the borrow checker, or a Haskell-like type system, changes the experience and the method of programming itself, forces certain patterns, and so on.
Using them also changes the experience, and there are common patterns to fake using them.
If you felt despair when you read the metaphor of totalitarianism and just want to get away from languages that want to grind an axe on you, and Lisp is not your cup of tea, then Raku and Perl welcome you.
But people have to get mad about the one throwaway comment at the top without even reading the next paragraph, let alone the rest of the article :(
Yes, you can't remove the constraints of Rust or Haskell (other than using unsafe, I suppose). But if those languages have the constraints that you want, then trying to write that as a DSL instead is... probably hubris. Also wasteful.
It would be a lot easier than inventing Rust, for two reasons:
1. You don't need to write a parser.
2. The target language for your DSL compiler can be Common Lisp rather than machine language without sacrificing performance.
Those two things make it absolutely certain that however much work it is to re-invent Rust in CL it will be strictly less work than inventing Rust from scratch.
There are other benefits to this approach as well. For example: if, after implementing your DSL you discover that you've made a sub-optimal design decision somewhere along the line, it will be a lot easier to tweak your DSL than it is to tweak Rust.
And in a world where Rust already exists, in most cases it's easier to just use Rust than trying to write "that kind of thing" as a DSL in Lisp.
Uh, how exactly does that work? Which CL impls can e.g. vectorize as well as rustc does?
One of the things people really like about Rust is its fantastic error messages. They're very helpful.
But, having decided you "don't need to write a parser" you're in a tough place for your DSL since there are problems where Lisp's parser will just tell your DSL programmers something went wrong here and leave them clueless as to what exactly is wrong in their program. Rust wouldn't have done that, but of course its compiler owns the parser too so it gets to handle all the corner cases correctly.
You could even write a FPS using OpenGL where shooting functions, or pieces of suctions removes them from the current compile job (or even deletes them from the source code entirely). In a macro.
† Or of course, to deliberately not emit descriptive error messages, e.g. whichever-compiles!
And yet rust has been invented, and it seems the CL version is hypothetical.
This surely cuts both ways: if you write a DSL that adds the constraints and provides the guarantees that you need, you've re-implemented big parts of Rust or Haskell.
Or maybe you only need a tiny fraction of the constraints and guarantees that they provide. But it's likely that you'll need more over time, and then you wind up with not just a re-implementation but a gradually accreted, rather than coherently implemented, one, and that's unlikely actually to provide the guarantees it's meant to provide.
Maybe. Or maybe all I had to do to turn CL into Haskell is implement the Hindley-Milner algorithm. I didn't have to write a parser or a compiler because I can just re-use those from CL.
Now they are reimplementing some of those things for performance reasons, but you're going to have a hard time convincing anyone that a lisp-based rust you wrote is going to be more performance than the one that exists today, so they'd need to make those optimizations anyway.
You are absolutely right that I'm squeezing a lot of content into the word "just". But I'm not doing it in a vacuum or out of ignorance.
It's such a wildly ridiculous claim though. It implies either
1. that the syntax of Haskell is the majority of the work, such that using a dsl would remove 90% of the effort.
2. Somehow a dsl for Haskell embedded in a lisp would be more efficient to work in, so much so that development would be an order of magnitude faster.
1 is absolute nonsense. 2 might be true in a sense. I doubt it, but even if it were, such a language would no longer be Haskell, so you'd lose out on the value you gain from it's syntax. The same is true for rust.
S-expressions are nice, but they aren't always the best way of structuring things, and if you're going to forego them to better embed another language, why constrain yourself to the lisp runtime?
The more I look at this, the less sense it makes.
You can create a DSL for a team working on a specific problem. Lisp is good at that. What I have not seen is a DSL that anyone else wants to use.
Someone builds a DSL, a small team uses it. It just has to meet the needs of that team. It doesn't have to be bulletproof or elegant. It can have all kinds of rough corners. It can take undocumented tribal knowledge to know how to use it.
The comparison to Rust is therefore almost completely misleading. It's like comparing Linux in 1992 to a commercial Linux distribution today.
There's a number of issues with that:
- I'd be missing all the optimizations that can be performed due to purity.
- There's more to Haskell's type system than just vanilla Hindley-Milner, and the implementation of it isn't particularly trivial. https://github.com/stylewarning/coalton is the closest thing and it's still missing a large amount of the type system.
- Doing the implementation would be a significant amount of work to get it to integrate well with the language, and it would be a layer tightly glued on top instead of integrated with the language. I've seen many good DSLs embedded in lisp, but a type system is hard to embed in any language because it changes fundamental semantics of the language. Typed Racket is a massive project and it's lacking things like ADTs.
- A major part of Haskell is the standard library, a good chunk of the semantics of Haskell people use on a day to day basis, like monads and etc, are a part of the standard library.
What I'm advocating here is not retrofitting Lisp but embedding DSLs within Lisp. In a DSL you have complete control over the semantics, and so you can do all the optimizations that Haskell does.
> coalton is the closest thing and it's still missing a large amount of the type system.
Note that this was written by one person. You have to distinguish between what has actually been done given the current state of the world and what would be possible if people made different choices. If a tenth of the effort that has gone into implementing Haskell had instead gone into implementing a Haskell-equivalent embedded in CL, that effort would plausibly be competitive with actual Haskell if not superior.
Not a fun codebase.
But say you do all that work. Congratulations, you now have an ecosystem of one person. To do anything in this language you need to reimplement a standard library plus third-party libraries, or at least write Rust-type-safe glue code to existing Common Lisp libraries. If someone wants to join your project, they'll need to learn your DSL first.
When you write a bespoke DSL for a problem already solved by a programming language with a sizable community you're giving up a lot of network effects. That is something the OP completely failed to account for.
It's a lot easier than writing Rust.
> you now have an ecosystem of one person
Do you think it would have even been possible for one person to write Rust? Imagine if all the effort that went into writing Rust -- and Haskell and Swift and C-sharp and Python and Java and Javascript all the other myriad balkanized languages that have come along since CL was standardized -- had gone instead into improving CL. Do you really think we would be behind where we are today?
Inventing a new and different syntax for every new language you design is a choice, not a technical necessity. The fact that languages have communities is survivorship bias. It's a bug, not a feature. In Common Lisp, it is possible for a single person to write something truly useful. In other languages, it is not. This is not to say that Lisp's lone-wolf culture is desirable; it's not. But that is NOT a reflection of any technical problem with Lisp. It's a political and cultural problem. And the first step to solving that problem is getting more people to recognize it for what it is.
No, but I don't think it would be possible for someone to implement an S-expression Rust dialect on top of CL either.
> Do you really think we would be behind where we are today?
Hard to say. The counterfactual world where everyone decided to build on top of CL is so weird it's hard to say anything about it. Also, "improving CL" is an unclear term. For example if you want to replicate all the benefits of Rust by "improving CL" then that means, among other things, you need an implementation of "CL" that can compile your Rust-equivalent subset/superset of CL to bare metal with no runtime (including no GC). That implementation will take advantage of Rust ownership and other properties thus will not compile regular Lisp code. If you have a separate implementation for a special dialect of CL with distinct static and dynamic semantics from normal Lisp, it is only very loosely still "CL".
> In Common Lisp, it is possible for a single person to write something truly useful. In other languages, it is not.
This is an absurd overgeneralization that makes you sound like a nut.
You're right. Allow me to rephrase: in CL it is easy to write a compiler for a new language because you don't have to write a parser or a back-end. It is so easy that it is a plausible task for one person to write such a compiler and make it truly useful, and indeed there are several extant examples. In other languages it is much harder.
Note that static type checking can be difficult to work with when using the prominent interactive programming style since the types aren't always wellhdefined and/or in flux while writing and adding new bits and pieces of code, or redefining types, adding struct/object members at runtime, etc.
But it's possible, yes.
Maybe it's cognitive decline from no longer being a teenager, or the experience of working on larger, O(100,000) line codebases with large teams, but nowadays I find a high degree of dynamism just exhausting. I don't want more expressive power, I want fewer nightmares.
A common problem I faced with Common Lisp is I'd write some module, then go on to write another module that depended upon the previous one, and get an exception through from the first module. Usually a type error. And I'd have to go back, context-switch, fix that, and climb back up to the other level.
With ML/OCaml/Haskell that is far less common. Being able to look at a module and say, "this is done and it's correct", is a very powerful thing. Confidence, rather than safety, is the primary benefit of static typing for me.
And I find that I no longer use the REPL. I've been working on a compiler in OCaml and for some reason Dune won't run utop (lol) so I've just not been REPL'ing and it's not been a problem. The code typechecks. Things work on the first run. If I change something, I get notified what needs updating.
The problem with interactive development is that it's like unit testing: it can prove the presence of bugs but not their absence. Type systems can eliminate large classes of bugs ahead of time.
Just so this is not entirely negative or depressing: there's something beautiful about how maximalist Common Lisp is. It's a big, messy language with every programming paradigm (and you can invent new ones) and different naming conventions in the core language itself. I was learning new things about Common Lisp years into using it productively. And I compared the experience to moving to a huge stately home, that has a room for the piano, a vast attic, a wine cellar, all manner of things, and then trying to move back to a shoebox apartment. Where do I fit the piano? CLOS, the MOP, the Lovecraftian beauty of LOOP and FORMAT: it's like a wild garden next to the (comparative) spartan minimalism of practically everything else. And it's beautiful.
It could be that you don't have a 'library component' defined in the project. Its dune file will look like e.g.:
(library
(name lib))
I'm pretty sure it's all correctly configured. But when I run utop #require won't find the modules.
I agree with the spirit of this line of thought: in general I favor pragmatism over zealotry, and I think it should be up to the programmer to determine how a tool should be used, not the tool itself.
However when we talk about something like the safety constraints imposed by Rust, it's more about citizenship than it is about the compiler being overly opinionated. By baking those constraints into the language, I can have confidence when I use third party code that it reaches at least some standard in terms of memory safety and thread safety. Much in the same way as a strict type system, those constraints free me up to not have to think about certain types of issues.
If I am writing code which only I will consume, I'm also free to use escape hatches to sidestep these constraints when I know they're not relevant.
I've written tens of thousands of lines of Rust by now, and I'm still not convinced the Rust model is the right one in terms of achieving its goals, but I think the approach is "reality based" with respect to avoiding some of the problems which can occur while developing software in collaboration with others.
Therefore, having the compiler deal with ownership correctly most of the time and maybe get in my way a bit some of the time when it's actually wrong (most of the time it isn't) is a tradeoff I'm happy to make.
But I'd argue most languages don't need ownership. GC is fine. Most of the problems we deal with in commercial software development are succinctly expressed in GC'd languages, and the benefit of using a language that explicitly tracks ownership is greater performance from being able to be closer to the metal.
Rust's ownership tracking is not only about memory safety. It can also help you with other kinds of resources whose ownership you can encode using the type system.
They could allow more relaxed GC'd normal types, whether default or opt-in, but affine and linear typing are genuinely useful properties fo reasoning about code, from both correctness and performance perspectives.
One needs to look no further than the string slicing problem for that to be clear. It's not an issue in Rust, but in every GC'd language you have to pick between:
1. eagerly copying the slice, which incurs additional often unnecessary CPU and memory costs, but avoids
2. every substring operation which isn't defensively copied being a potential memory leak as you're slicing 3 characters off of a 5MB string on every HTTP request and storing that in a global map, which turns out to store the entire 5MB string you got it from
Or some even more complicated magic where you only perform the copy if the substring escapes, at which point your performance and memory characteristics become wildly unstable and the mere act of extracting a function can bring the entire system to its knees.
I'm working on this: https://github.com/austral/austral/
The idea is to start with a simple, easily understood linear type system and then add borrowing, but without going too far in the direction where the type checker becomes a tower of heuristics and conveniences so that programmers can write normal-seeming code which magically typechecks.
So you can learn how to write linear code from reading a set of linearity rules, rather than from trial-and-error against the linearity checker.
Have you ever thought about putting a simple code example right in the README? Maybe it's too early days, but I know that's always the first thing I'm looking for when I stumble across a new language.
Of course there's tradeoffs involved, but I think there are multiple approaches to achieve safety and user-space ownership constraints are only one of them.
Many forget that MSIL is like LLVM, having all semantics to compile C++, alongside GC.
Some of those capabilities weren't exposed to other .NET languages, however since C# 7 they have been improving it.
As of C#10, there is little left to do versus Modula-3 or D.
Borrow checker only helps dealing with concurrent access on the very special case of multithread code accessing in process data structures.
Anything else that follows outside of this, like concurrent access to external resources, or via OS IPC, there is little help.
So, the thing this gets you, as the Nomicon explains, is you're data race free and so you get to have Sequential Consistency (in Safe Rust) within your program.
After decades of computer programming, our experience is that humans need Sequential Consistency to think about anything that's too tricky to just write down a complete list of all cases. This is terrible news if you write machine code, since your modern CPU doesn't bother supplying Sequential Consistency in favour of going faster instead, but it's also very bad news in many languages (C++, Java, Go) where the promise is SC/DRF but you're on your own to supply that necessary data race freedom (in fact in C++ it's worse, if you can't supply data race freedom you get Undefined Behaviour).
The traditional (and especially on Unix, well rewarded) solution is to give up and only write serial code. Then your program doesn't have concurrency, so it will be Sequentially Consistent. When the average computer didn't have pre-emptive multitasking this felt like a pretty sensible way to write programs. Even once the average computer did have pre-emptive multitasking (and so threads are a nice win for some problems) it did not permit simultaneous execution, most programs weren't slower for being serial. But today lots of people even own a smartphone with more than one CPU core.
So hence Rust's Fearless Concurrency. Instead of an hour-long tutorial full of caveats and generalisations (to write parallel algorithms in C++) you can confidently write your Safe Rust with concurrency and the compiler will reject any fumbling attempts that introduce data races. Instead of needing to call Sarah the grizzled concurrency expert for any changes to functions in scary-but-faster.cpp you can let Bob the new girl modify the code in faster.rs, confident that either Bob's changes are actually safe or you'll spot the awful mess she made trying to pacify the borrow checker during code review.
If you have multiple threads inside the same application talking to the same database, and modifying tables without proper SQL transaction blocks, anything goes and the borrow checker doesn't help one bit.
Additionally, if those multiple cores are being used by multiple processes, microservices style, there is also very little that it can help to fix IPC data races to shared OS resources.
But Rust will enforce constraints your model reflects. For example it's common in embedded computing to reflect hardware resources as singletons. If the firmware_updater is using the only SerialPort then it isn't available for my doorbell_noise_generator, I can't just make another one and ruin everything by reconfiguring the serial port that's currently moving firmware program code.
Rust has no idea what a SerialPort is, but the programmer provided no way to make any more of them, and the only one that existed is owned by firmware_updater right now, so too bad you can't get one for doorbell_noise_generator and the device doesn't get bricked by a user pressing the doorbell button while doing a firmware update.
If your application needs database consistency beyond what your chosen RDBMS actually promises with non-transactional updates, you should definitely provide Rust access to that database only via transactions that preserve the required consistency. If your RDBMS is so feeble it can't cope with more than one transaction in flight, you may need to make those singletons too. The ownership model is enforced by Rust and you'll do fine.
As I said, humans can't cope with anything other than sequential consistency at scale when reasoning about systems. If you've built a complicated "microservices style" system that doesn't actually have this, the humans supervising it don't understand how it works and sooner or later (but likely sooner) it will do something that is entirely outside their model of what's possible and they've no idea how to fix that.
Remember, Rust is not an innovator here. It's applying lessons understood in academia - to an industry which had been infested by Real Programmers and couldn't understand why now everything is on fire.
People talk about the problems colored functions when it comes to async, but in my experience the real issue you run into with Rust is that once you need to introduce an explicit lifetime into a data structure, the complexity of everything touching it starts to balloon out of control.
This creates all kinds of complications, creates problems with automatically deriving the future trait, and leads to workarounds where people end up passing id's around everywhere instead of references in a lot of cases, which can effectively be a way of circumventing the borrow checker.
I don't know what the answer is, but sometimes Rust feels like it's an abstraction or two away from really solving the problems it wants to solve in an elegant way.
I feel like it's not unreasonable to say that some parts of rust are impressive because they make hard things ergonomic, and others - so far - are mostly impressive because they make really hard things possible at all.
Or: I share your intuition, but assuming there even -is- an answer, I think at the very least rust has provided a great service in figuring out what the right questions are.
Well, a reference (or a pointer) is just an index into a giant global and mutable array of bytes, which is pretty crazy!
Passing around indexes into specific arrays of objects is a lot less crazy. The indices are stable even when you reallocate the array, for example. Think of them as offsets, rather than pointers.
It's a perfectly workable approach, but passing around offsets feels like I'm breaking the contract a bit. The compiler doesn't know which index goes with which data structure, I'm just asking the compiler to trust me that I'm pairing them correctly.
Also it tends to be more boilerplate than just having a direct pointer stored inside the data structure.
Don't get me wrong, I understand all the problems with pointers, but it the UX is better in a lot of cases.
Another angle on this for a language like Rust is that Rust is designed the way it is because it’s favouring reality over programming ideology, distancing you from the thing you’re trying to do when it’s something that doesn’t map to the reality of how computers work very well—and yes, that reality does end up leading to ideology, but Rust is the way it is because of reality more than because of ideology.
But then Lisp! Why, Lisp is the epitome of programming ideology over reality! Ask me to name a “big idea language” under the provided definition and the first language (well, family of languages) that springs to my mind is Lisp. It’s not just unopinionated, it’s aggressively unopinionated, which is an opinion, an ideology (to say nothing of the doctrine of S-expressions), and one that flies in the face of how computers actually work; so that you pay at least a performance price to use such languages, and often other prices too, like with threading, as ekidd’s comment mentions.
There’s a reason Lisp machines died.
The very property that makes it liberating - that you can instantly try things out and see mistakes - allows you to elevate up a level or two in complexity of what you can do. But the future readers of your code don't have this advantage.
While you can apply discipline and use it to work to find the simplest, clearest solution, there is nothing that forces that and you can just as easily arrive at something that looks and works like magic to someone coming to it fresh.
I find the repl encourages me to iterate quickly and when i set out to solve a problem, one of my goals is to express the solution in a nice way.
The repl just reduces the cost of me trying different ways to achieve that. Whether i leave behind horrible or nice code isn’t really a factor of the repl, it’s a factor of whether i started with an intention to write understandable code. I usually do and the repl’s instant feedback and state retention makes that vastly easier.
Sometimes i don’t set out with that intention though, i frequently have to process lots of text and regexp is just the easiest way for me usually - even though i accept it’s utterly impenetrable to many once you go beyond 10-20 chars or so in an expression. No repl involved in producing fairly impenetrable code.
Similar to what you are talking about here, I worked on one of the problems for a while. Not in a REPL, but in a similar manner. And I arrived at a solution that is not advanced or anything, but which I was quite satisfied with. It occurred to me then that my solution to the problem I was working on really only made sense because I had worked through the problem.
For this reason I left a printout of two of my hash tables in the code. Because when you see what data they actually have in them it becomes quite obvious what the code is doing. Whereas without those two printouts you’d have to “simulate” in your brain what the code is doing.
And so for that reason I left the printouts in my code.
But this also ties into a more general problem in software development which is that even though our tools are becoming really really good and our computers really really fast there is still a long ways to go in terms of how deeply and how completely we can really see what’s going on.
I was thinking about this recently when I was at the dentist to pull a tooth. It was not a fun experience but even so I saw that the dentist had both the skill and the tools for making the operation. And in particular the tools he had at his disposal allowed him to understand and to operate on my teeth. And that got me thinking once again about the lack of visibility that we have when we develop software.
Future readers also don't have access to my brain state when I'm implementing the program. Whether in Lisp or in Haskell or in C or in Python. This is kind of a nonsensical point. Iterating over a solution in Java and in Lisp it amounts to the same thing: code in a file. There's nothing keeping you from documenting.
Emacs makes doing this too easy, though. I prefer slimv which has no extra repl buffer. It forces you to edit the sources and invoke REPL with snippets from there - ideally, whole file stays compilable all the time. Or at least to use a scratch file, to later get back to and refactor. Any programmer bitten by their own unreadable code should learn to do that and resist siren calls of repl patching. But then, unreadable code is possible in any language.
I suspect the way you're working with slimv is at least somewhat comparable in terms of making getting that part right the path of least resistance, which is itself a form of developer experience ergonomics.
Can you provide an example for this? Intuitively I'd say you are right, but, OTOH, even after 20 years of CL-hacking (which probably does more REPL-driven development than any other language), I cannot come up with an example...
This is (usually) a sign that I forgot one function or macro in the code-on-disk, that I hand-chased in the REPL. In neither case was it hard to rectify, but it did leave me a bit confused. And very thankful that I had not quit the main dev image.
After that, I have explicitly added "add a test suite, run test suite in fresh image" as a technique to keep that from persisting for a long time. And simply doing that seems to keep me subconsciously aware of the problem and not making it appear in the first place.
Now, sure, you can write a mutating statefull mess using the repl as well, but most code I have come across in CL is not like that, except for maybe the odd times you find java-in-cl.
It is not the same thing, of course, but it is tangential. Pure functions is something you can write in any language, but languages where you so easily can develop/try smaller chunks make it a particularly good idea. Trying a piece of Java code usually means running a lot more code that sets up your state for you. How I, and most people I know, use a lisp repl does not.
Misconception is way too kind a word. And it's quite ironic (though not necessarily unsurprising) seeing the article's namecalling of bad faith as it engages in paragaph after paragraph of bad faith and being plain wrong.
Well, for one, it's hard to start a good-faith discussion about a tool by calling it totalitarian in the introduction paragraph.
I apologize that the above paragraph does not actually convey the humor it was written to explain.
Unfortunately, no one can be told what humor is. You have to feel it for yourself.
My point is that making fun of something you don't like, however light the fun is, is not a good faith technique. Now, that Doesn't mean bad faith discourse can't be hilarious, Conrad Barski had made that same joke quite well in his comics.
It's just hard to make it the start of a honest discussion on the topic.
It's striking how preferences in programming languages seem to mirror preferences in communication style: You demand that the writer express himself in your preferred style, and if he does not, you accuse him of not writing in good faith. The writer, on the other hand, writes in a liberal style, full of metaphor and references to philosophies outside of computer science.
You only tolerate a narrow range of expression, while he welcomes wide and varied expression. You take offense rather than seeking to understand the intended meaning, while he takes into account others' interpretations and tries to meet in the middle: https://hyperthings.garden/posts/2021-08-30/freeing-your-goa...
There are some life lessons to be found in these exchanges. Postel's Law is not well-followed anymore.
In fact Haskell has a decent REPL, and you can (if you really want to) explicitly allow IO outside the IO Monad, or skip the type checker, though I find them more of a moral compass than an impressive dictat. I don't know if you can crash straight to the debugger like in common lisp though.
But better to just block 5 mins in your calendar and try it: https://github.com/PEZ/rn-rf-shadow
Don't play with this if you like hot-reload or hot-refresh features in react or jrebel or whatever. You will forever see these other approaches as lame in comparison... speaking from someone who was previously very happily ignorant, living my best life on the JRebel side of the fence.
It's pretty cool to see music, graphics, and game behaviour change live in response to code entered into the REPL.
You can get stack traces with access to variables via :set trace, but because of optimizations and lazy evaluation they are similarly jumbled as async code debugging.
Haskell has some nice advantages like `:r` for reloading, similarly you can reload a dynamically linked binary and the old version is only unlinked after all references to it are gone. There is a significant difference to dynamic languages, though, because having different versions in memory leading to incompatible types would be a lot more common with a partial reloading workflow.
That does stay on the happy path, though, and CL has a much better REPL experience with regards to debugging.
I think type systems and other restrictions are really good precisely because they help me better trust and collaborate with others.
It's not my cup of tea, but I respect him for doing something different and taking this thread out of the haze and to its conclusion.
This statement doesn’t even slightly ring true for me. These compilers are more like infallible personal assistants who keep track of all the crap that I don’t want to. The Haskell compiler in particular has never suggested I stop doing something that didn’t turn out to be a bad idea under more careful consideration.
The rust compiler is more restrictive (mostly due to lack of ability to express some types, like quantified or higher kinded types), but still is fundamentally working on my behalf.
That's great until someone else tries to use your program.
I'll take the compile-time safety checks, thanks.
Basically they took random JavaScript bugs and annotated them with Flow and Typescript type hints and then checked whether the bug was detected.
Pretty clever methodology really.
So when GHC says "you're doing side effects outside of IO", it's not trying to harsh your vibe. It's explaining to you the inconsistency between your program and the "type" (description) that you've written for it. You might not want to have to explain yourself to the compiler, and that's fine -- go write in an untyped language instead. But it's hardly authoritarian to offer a different way of programming.
(Keep in mind that typechecking meets many of the same needs that debugging does. For someone who already tries to typecheck in their head to ensure that their programs are correct, having an explicit, external typechecker can be a relief. That might not be you, but it's at least some people; when I program in Lisp I'm always running into problems that a typechecker would have solved at compile time, and I find that frustrating. To each their own.)
Two systems I’m familiar with:
* In Flutter, you write code in the normal way, in an editor or IDE. If you want to change code while the program is running, you edit code in the normal way and hit a key to reload it. This works when stopped at a breakpoint. But the changes you can make using hot reloading seem somewhat limited; you couldn’t do a major refactoring.
* In an Observable notebook, you edit a cell and then any other cells that depend on it automatically run, like a spreadsheet. You don’t normally use a debugger, though the browser’s debugger does work. In some cases when you’re using advanced features, you might need to reload the page, but usually you don’t.
How do people work using Common Lisp? In other languages? In particular, do you write code in one place (using the repl) and copy-paste it to another place?
Sometimes I work in an Org file, sometimes in a lisp file directly. In both cases I will C-x C-e (eval-last-sexp) or run blocks, or the whole file (depending on Org vs lisp file). It is copy and paste on steroids (same for slime-eval-defun were you can be working deep inside some large function and update the whole without changing your view).
There are some examples of developing in the debugger in [0] around 55 minutes (linked). This video also has great examples of other crazy interactive development workflows and is worth watching in its entirety if you are interested in this kind of thing.
This seems like it would make the most sense for event-driven code. Evaluating a script is another kind of event. Many systems have a way to execute commands.
I'd guess this style of development encourages people to structure their systems as essentially command interpreters, even if the commands normally come from a UI.
The interesting bit will be deciding how the data structures that persist between commands are implemented. For example in server-side web apps, the shared state is often a database, so state is external to the process being debugged. The architecture limits what you can do directly, but you can try out SQL queries.
I'm sorry emacs fans, I've tried multiple times, vanilla, with distributions... it just doesn't work for me.
[0] Of course, that doesn't apply if you're building the program entirely in memory, such as Smalltalk with its image-based programming environment.
At its core it's just:
var obj = {}; obj.a = 10; obj.a; // returns 10
And with browser dev tools you can debug very efficiently.
I would agree that the interactivity of the REPL is not at the same level as CLisp.
The only other runtime I have found that has a similar absence of pain is Emacs (though Emacs has its own separate warts unrelated to friction at the repl). I think there is a deeper disconnect between having something that looks like a repl and a real repl which is that for decades Emacs didn't even have a command line style repl, because the repl was everywhere, hidden behind C-x C-e or M-:.
A point of confusion I think is that repl in the lisp world has come to imply much more than that somewhere there is some code that looks like (loop (print (eval (read)))). That level of sophistication of the repl concept was archaic well before common lisp, but the term continued to be used and evolved to signify a runtime that could support interactive development.
For many of the other languages the issue isn't with the form of the interface (almost any language can have a prompt, come up with printed representations that can round trip through their printed representation, etc.) it is that their underlying runtime can't support the kind of redefinition that is needed to avoid the creeping insanity that can arise when code becomes subtly out of sync with data e.g. in Python classes and instances.
Computers are faster now, so the workaround is "restart the runtime" and users hardly notice. Therefore no one bothered to implement what common lisp has. Given how long a cold boot took back in the 70s and 80s, avoiding the restart at all cost was a top engineering priority. Lisps evolved in a significantly different and harsher selective environment than many of the runtimes that came after, and as a result had to solve certain fundamental problems that are practically invisible for users today.
In the time since I wrote the referenced comment on reddit I have also discovered the amazing power of save-lisp-or-die. At the time it hadn't quite clicked for me. I'm guessing that the author has had a similar experience given the title of the series so I'm looking forward to reading more in the future!
In the mean time I also learned a bit of docker, and what I find amusing/sad about it is that slad is basically docker without all the fuss. Sure it can't bring a whole unix system with it, but it is a couple of orders of magnitude smaller and significantly less complex.
The opinion of Rust and Haskell is precisely that making mistakes is bad, regardless of your "provocative" opinion that it might be good, and that the language should be a https://en.wikipedia.org/wiki/Poka-yoke against certain categories of mistakes that have been found to cost the industry billions of dollars in failure.
(What do the Clojure-for-web-services people do? Presumably that doesn't drop web requests to an interactive debugger, or does it? Or is that irrelevant because this is only concerned with Common Lisp?)
This kind of advocacy seems to be endemic in LISP and FORTH: a tool produces great results when used by one or two idiosyncratic "genius" developers working in near-isolation on problems of their choice. It tends not to work nearly so well beyond that.
Not precisely what you're asking but I have, very occasionally, REPLed into remote clojure servers and rewritten running code live to fix time-critical bugs in production. But I'll admit i'm unsure whether this is an argument for or against Xtremely interactive development.
Darklang does offer something they call trace-based development where unhandled requests basically do initiate an interactive process to write handling code. I'm under the impression though that this is not intended for production time.
In general I'm loath to make any generalization about what kind of mistakes are good or bad and when. Luckily we have a language landscape which lets people make up their own minds.
While Dark does support the general idea of doing live code in production, it's not a Lisp dialect and is very much a functional language.
My experience is that if you make the edit->run->edit loop tight enough, you can recover a lot of the guarantees available through static checks via a couple of alternate techniques and, for some of us, the resulting workflow is a lot more pleasant way to spend the day than tweaking the code up front until the compiler accepts it.
In CL the interactive debugger is only one option of how to handle uncaught exceptions. You can set it to just crash or log or whatever you want for production use case.
In Clojure it defaults to the host runtime default handling, so like in Java it'll throw all the way to the thread and kill it. Unless you explicitly handle it somewhere.
In JavaScript browser, it'll just get logged in the browser console.
And I'm not sure what NodeJS does, maybe it crashes the process?
But I'd say Clojure is more like the OCaml of Lisp, it nudges you strongly towards pure functional programming, managed memory and thread safe behavior. But it isn't strict about it, which allows you to still get a fully interactive development workflow.
It's a bit like how in Python private methods are a convention, turns out defaults and conventions tend to be followed pretty well by programmers. But having it not be strict means it can accommodate certain weird use cases when needed.
I like to think of it like where Haskell and Scala have safety guarantees, Clojure instead has a safety protocol. In practice both seem to yield on average similarly correct programs.
(Of couse you can still have expensive failures caused by bad logic, but I think in security bugs alone the cost of C/C++ style nasal demons behaviour has been bigger than other bug categories)
- Supports tab completion, shortcut keys like ctrl-a/ctrl-e and probably more that I don't ever use.
- Top notch support for unicode and supports color (IO.inspect with [color: true])
- EZ access to docs (Just type h in front of what you want to know about)
- You can recompile your project without leaving the repl (keeping whatever state you have)
- You can access the history of what has been returned using the v function
- Remote shell support (type `h IEx`) to learn more.
---
One thing I don't like: If you get mixed up and forget how many braces or parenthesis you need to close, you get stuck. I usually have to quit iex to get out. There may be a better solution to this that I'm just not aware of
When that happens, I simply enter 'iex> end' which would return a SyntaxError, and start over but without having to restart iex.
- it seems some core lisp/Slime features are not there: compile one function independently, get type warnings and errors, goto source definition working OOB, no function signature (?), find who calls a function or macro, who sets a variable, and then no interactive debugger, no stepper, no inspector… ?
They also linked to the more questionable entry on wikipedia rather than the more authoritative one. Compare https://en.wikipedia.org/wiki/Bad_faith vs https://en.wikipedia.org/wiki/Bad_faith_(existentialism)
> These "big idea languages" tend to assert a kind of "programming ideology" over reality, and tend to, in my opinion, distance you from the thing you are trying to do, often, they claim, "for your own good".
Except they don't pull these programming rules out of thin air, they're actually widely acknowledged as the best practices in the domains they're targeting. It's like complaining that SQL database engines don't let you hand-write custom query plans in your queries, 'for your own good'. Well yes–that's exactly the point! They've been finely tuned over decades of research to know how to do it better than humans.
> They are like the neighbor's kids in a totalitarian regime who will report you to the secret police when they think you're not quite as doctrinaire as you ought to be.
Wow! Comparing a technological tool to a fascist regime, because that's totally accurate and appropriate! /s
> Haskell compiler might be heard to say, "What are you doing? You are a Haskeller and should know better!" ... complains the Rust compiler. "Are you insane? You are a Rustacean, you don't do that kind of thing!"
Yes, of course, compilers are exactly shrill, screaming drill sergeants trying to break you down and indoctrinate you. By contrast, doesn't Common Lisp seem so mild-mannered and gentle? Of course you'd never want to use anything else!
Now I know it would be a little extreme to call this argument 'gaslighting', but I honestly can't think of a better word.
> Encouraging you to avoid something, however, is quite different from banning it outright.
Yeah, which is why Rust and Haskell actually don't ban it outright, and in fact why almost every language has 'escape hatches' that allow programmers to do whatever they want, on their own recognizance.
> When you are programming interactively, you are not so much writing code as sculpting and structuring live computer memory.
Great, but once I'm done with that, I need to actually lock in the shape of the sculpture so that it can be deployed with confidence.
> True interactive development is also about programming in such a way that your program never crashes. Instead, whenever an unhandled condition is raised, the program starts an interactive debugger.
Great if you're planning to keep a watchful eye on the program constantly. Not very helpful if the program is supposed to run unattended.
> Hell is Other REPLs.
Clever, but even other REPLs allow you to do the crucial bit–i.e. interactively explore an API. And after that, the best languages even allow you to lock in your findings and deploy with confidence!
What CL development I've done, it was pretty much "reload it and rerun it" in terms of a development cycle. Mind, these were not large programs. But there was enough global, shared state that needed to be reset that, most of the time, a simple tweak to a function wasn't enough to right whatever wrong was involved. And the reloads weren't arduous anyway.
Sure, for "little work", "local work", tweaking a routine, doing simple tests and such in the listener. It was fine. Very quick turn around.
But when fixing something that was larger in scope? Reload it, rerun it.
I also never "got" the restart and condition systems in CL. Part of this is simply my background, today mostly being in Java where production debugging is doing archaeological digs on deep stack traces.
I get restarts in interactive systems. But my systems were not interactive. They were server based systems, processing zillions of requests. I never saw the utility of a restart on a headless system. I could not imagine a connection stuck in a restart just waiting for me to fix it, debug it, correct it, continue it, or abort it. In contrast to just logging the torrid details of the failure and path to it and using the information in a post mortem.
Do folks get 3am phone calls to open up a server and dig through restarts? That never made any sense to me. On paper, it sounds great, I just never saw any realistic way it would ever apply in any of the work that I did.
Are there times it would have been nice to log in to a server, tweak a piece of code, and let it continue on? Changing the tires of a car on the road? Sure. Occasionally.
Mind, that could just be habitual. Since to me it was a novelty, and one unavailable to me, perhaps I simply don't miss it. Yea, it makes sense when hacking deep space probes. But a generic remote web service type of application in production? To me, not so much.
The idea of hot patching a server is amazing and frightening at the same time. How was the patch tested, do you commit it to VC first, before cut and pasting the DEFUN in to the listener, etc.
The same applies to Smalltalk. The idea of sending out an application that faults and drops the user in to a restart and debugger. What can they do with that? Call me up and talk them through it? I'd much rather them send me the log file with a post mortem I could use. I'm sure ST has ways of capturing stack traces in log files, but, casually, nobody talks about it.
So, I'd love to hear stories about what I'm missing. What applications folks were doing that were leveraging these aspects of the CL experience. How they manifest on system with any transaction volume (you know, a few trx per second). How one uses restarts in a web shopping cart in production.
Make no mistake, in Java, with the applications servers, turn arounds can be pretty long. But, similarly, with code structure, units tests, etc. turn around can be very fast. Make a tweak to the code base, repeatedly run an isolated unit test until it works, then run the larger suite to see what else you broke. That can be quite fast. Not "interactive", but...fast. "Close enough".
I agree that function-level recompilation of definitions feels alarmingly undisciplined, borderline irresponsible in a production context. On the other hand, in my $work, I can recall sufficient instances where being able to compile in just a little extra targeted debug logging to a running process would've turned completely baffling production issues into something easily explained, sparing a lot of time and hard thinking trying to infer via code review the root cause of issues that no one was able to reproduce internally, and provide much greater confidence of fixes.
It's worth mentioning that on the Lisp machines, the debugger was a first class component of the user interface in a way that might feel very foreign to people accustomed to the Unix command line. It was just a way of inspecting backtraces, it was a essentially a form of interaction where the application could present a problem situation and offer the choices how to proceed.
With smalltalk the user could send you the image in its current state which would provide a lot more context than just a stack trace and a log. I definitely would prefer to pickup a running environment at the point of failure than a log when solving a bug. I cannot imagine how one would keep up with security on a setup like that though.
The "image in its current state" had been saved with an open "Walkback Window" showing an exception.
I opened the debugger, identified and fixed the problem (iirc without needing to reshape any user data with become:) and resumed the exception; saved the image and change log to the micro floppy disks, and snail-mailed them back.
Apparently the users picked-up their work where they'd left-off, when they sent it to me.
"Chapter 8 Debugging"
http://rmod-files.lille.inria.fr/FreeBooks/SmalltalkVTutoria...
No, back then it would be a 3am pager beep; triggered by Smalltalk resumable exception, possibly when someone was on-call (in a bar).
> … nice to log in to a server, tweak a piece of code, and let it continue on?
Critical not "nice".
Trade reconciliation done upstream on mainframes took many hours (not enough time to do over) and that upstream processing was sometimes broken when it was changed, and then those errors were caught and mitigated by downstream software — because there we could "tweak a piece of code, and let it continue on".
This is cool for the solo programmer.
Teams and organizations need some amount of standardization if they want their programmers to be able to maintain each others code. At that point, none of the coolness of Lisp remains, and you're better off using a less dynamic language that imposes more of a standard style.
Who doesn't come back to their program after 6 months of doing something else.
With the Common Lisp standardization, a subgroup was given the task to decide on a standard object-oriented extension or to develop their own. After several years of work, a core group with a large group of supporters and implementors then defined a standard language extension to integrate object-object-oriented programming in Common Lisp - the Common Lisp Object System, which is now widely used in Lisp.
I think these languages are focusing too much on a few ideas while ignoring the rest. For example if I'm writing a Rust program which launches missiles, then I can still accidentally launch these missiles in safe code. A GC'd language would still provide good memory safety guarantees while allowing me to focus more on the problem at hand (not distracting me with type errors), and thus it could be safer overall.
Problem is people trying to push it as one true solution. But that's a human problem.
> Linked lists are as niche and vague of a data structure as a trie. Few would balk at me claiming a trie is a niche structure that your average programmer could happily never learn in an entire productive career -- and yet linked lists have some bizarre celebrity status. We teach every undergrad how to write a linked list. It's the only niche collection I couldn't kill from std::collections. It's the list in C++!
> We should all as a community say no to linked lists as a "standard" data structure. It's a fine data structure with several great use cases, but those use cases are exceptional, not common.
I have never in my professional career used or to my knowledge relied upon a singly-linked list, to say nothing of a doubly-linked list. That feels like picking something to be contrarian, not because it exemplifies a good case of where Rust is too strict. Just use a Vec? It's way more performant anyways.