Weird Languages
paulgraham.com
paulgraham.com
"What can you say in this language that would
be impossibly inconvenient to say in others?
In the process of learning how to say things
you couldn't previously say, you'll probably
be learning how to think things you couldn't
previously think."
Criticisms of this essay in comments so far have seemed to fall into these buckets:
1. Lisp? Macros? Boo. (Not novel/too weird/etc)
2. Languages are solved, not important; I build systems.
3. I don't like the tone of this essay.
I've always been in the pg camp on this one. Sure, systems are the thing, we can't be purists, etc -- but saying that languages don't matter, for us, would be like rocket engineers saying that materials don't matter.
If you're building a table, you can make it out of almost anything; a table is a solved problem. Wood, carbon fiber, steel.
But at the limits of what's possible, like a rocket, the material's properties, limitations, and weaknesses in extreme circumstances matter.
You can definitely imagine that important things are happening, in the maker's mind, when they start working with a new material, seeing what others are able to do with it, seeing how the making process is easier, or what steps are unimportant/unnecessary now vs. in their usual materials. And that informs their thinking, going forward, about what's possible in systems they might build.
This argument always gets funny when sufficiently complicated systems start themselves looking like languages (and their implementations).
The thought I have related to this is that languages are much like GUIs in this regard. They enable a certain process using some built-in features, but it's next to impossible to abstract over those feature to optimize further.
And I think that is why I stay with a simple language that gives arrays, structs, and function calls - basically tools to manipulate data / memory, I'm not sure that more features are all that useful to write "well-compressed" programs (but I can imagine that it's often hard to see why certain useful seeming features will be detrimental down the road).
Two years down the line people start to realize that steel actually isn't perfect either. But wait, now someone made a carbon fibre table and we need to break down all our steel tables and burn the few wooden tables left and rebuild it all in carbon fibre! It's the future, after all!
Of course, eventually people figure out that wooden tables are actually quite good for a fair number of use cases and we've come full circle.
I'm just tired of the churn-and-hype cycle and suspect a lot of people have roughly similar feelings, which is expressed in the "languages are unimportant; I build systems"-sentiment. Less and less of modern software development seems to be about actually building interesting things to solve real problems.
Programming languages don't churn nearly as fast (and some degree of existing churn can be attributed to the business model of a PL funded by a company - they have every incentive to push it, no matter how little improvements or different concepts the language brings.)
I’ve been pushing myself to learn elixir these last few months. This is, by the comments of many, more of a niche language. I like the language’s novelty and it has caused me to think in new ways. So Paul is righ in this pint.
But when I want to send an http request, there must be at least 10 choices on hex to choose from. The one that is a hit this year will be passe’ next year. When I watch the traffic in the Phoenix slack channel, the “stack” of libraries combined seems overwhelming. The same goes for people talking about front end apps.
https://erlef.github.io/security-wg/secure_coding_and_deploy...
It's just not fun to be in a constant state of package fatigue, especially when a lot of semi-popular libraries are maintained by 1 person. If that 1 person leaves that entire bit of functionality becomes a liability to your application because it's unrealistically demanding to be a solo developer or small team and be responsible to maintain a bunch of libraries that become unmaintained but then also build your application.
This is IMO especially true with a language as powerful as Elixir with macros. Picking up and reading someone else's code can either feel natural or be really hard, to the point where looking at it doesn't feel like the language you know. This makes it doubly hard to read 3rd party library code because you're learning the other person's code and have to mentally deconstruct a lot of advanced patterns if they use them. I don't consider myself that good of a programmer but there's a lot of times where in Elixir, phrases like "this is impossible for me to understand" come up when I've written hundreds of thousands of lines of code in Python but never thought this. I sometimes had the "what am I looking at?" feeling with Ruby too btw.
As for library availability, this is a problem that doesn't come up as often with more main stream web frameworks, at least not in the Python and Ruby eco-system. Sure things go in and out of fashion but both languages have frameworks that include more batteries included or the community has gotten so big that there's a bunch of independently developed libraries that most of the community got behind and supports it so well that it becomes a no-brainer to use it.
So that's what keeps me on the fence and I think some of these problems can be solved with time, but time is a scary thing because waiting a few years feels like an eternity and the outcome after waiting a few years might be waiting a few more years. At some point you need to build whatever you're building and run with it without looking back.
But I think this really comes down to personal preference. I used to think I wanted unlimited choice and an endless world of possibilities. I don't. I want to build web applications quickly using the most straight forward code I can think of and get as many batteries as included as possible while leaning on / contributing back to a massive eco-system. I don't want to be on the front lines of building a new eco-system. I still do think in the very long term (5+ years maybe) Elixir could be a good match for me once there's a lot of libraries that have hit critical mass.
There's plenty of people using it successfully now, and I'm happy there are people who like being closer to early adopters and setting up the initial ground work for an eco-system.
Is it a mistake to spend time learning Elixir?
It also depends on the app you're building. For me I was building a very large app (a video course platform with multiple payment gateways). One limitation I came across with Elixir is there's no official SDKs for Stripe, PayPal, Braintree or Paddle but there are SDKs for most other popular languages. So if you plan to interface with a number of APIs, chances are you're not going to find an official client for Elixir unless the company happens to be using Elixir or are happy to support it (such as Mux).
It could be challenging and time consuming to have to write both your app level features and the underlying libraries. If your goal is to get a web app launched quickly to test it out and you already have experience with Python, Ruby, Node or PHP it's kind of a hard sell to use Elixir. It's mainly because other languages have way more community libraries and frameworks available that cover common web feature use cases. Plus there's a lot more community resources around blog posts, videos, books, courses, etc..
There's also an interesting divide right now in Phoenix around using Live View or not and if you go down the LV route there's very few resources beyond the docs, especially now with LV 0.16 coming out which completely changed the templating system and how components work. It still feels like it's in a pre-1.0 (because it is :D) semi-experimental state. As far as I know there's also no really big success stories with LV. I don't know of any big sites who are using it all-in to drive the entire navigation of their site and all routes.
The above continues to fall in line with my overall feeling of having choice fatigue with Elixir.
With that said if you're interested in it, maybe try to put together a proof concept that focuses on the hardest part of your app and if it works, make a judgment call based on your personal experience. It's possible you'll have better luck than I did. Programming language selection is a very personal choice, especially when you get past libraries choices and start writing code. If Elixir clicks enough for you at a fundamental level maybe it'll be a better choice even if it means having to write more libraries.
When I look for help in the Kotlin slack, the responses are just a firehouse of “you need Zanzibar, no Algolly, try Firetruck, do Conduct, you might want xWyze…”
But yes. It's true http clients have churned quite a bit. Other things are settling better, for instance Ecto, Phoenix, Jason are very popular and stable.
So much of the fetishization of everything "new" comes from a combination of wishful thinking and mental laziness. Especially in larger organizations where decisions are made far from where the work is actually done, there is a tendency to think we can solve problems simply by embracing the newest management fad, enterprise software, etc. It's kind of like the way people think they can get organized "if only they had the right trendy organizer or todo app".
I get the impulse, we desperately want to believe that our hard problems can easily be solved by just changing one thing or buying something. This is the crux of most modern marketing, in fact. Unhappy happy with something hard, just by product or service X.
Older workers tend to know better than others that hard things are hard and that layering some shiny new tool or framework on top of crappy process or a broken business model will not change anything. But saying that out loud, even tactfully, is a very inconvenient truth that will make most managers resent you, and probably label you "resistant to change".
But then, I'm not known for fighting against new solutions either, so that experience is probably in a disjoint population. Anyway, I imagine you say that from the position of an "old wizard", but that position is as dependent on a receptive audience as it is on a communicative "wizard".
I don't actually care though, just I can see this possibility.
My comment above focuses on what the stoics advised to focus on - the things one is in control of changing. In our case, old wizards could control their attitude.
Maybe it's just me and my possibly-unreasonable infatuation with the Sapir-Whorf hypothesis (SWH) but I've come to think it also applies to many things in life, PLs especially so. "Weird languages" aren't valuable because you can deploy it in production, rather because it will help you recontextualize a problem (in SWH-parlance, it helps you see the world differently).
You can also say, SWH is an academic formulation of the beloved programmer motto "When all you have is a hammer, everything looks like a nail". If all you know is an imperative syntax, all problems look solvable by breaking them down into a series of steps/procedures.
In my almost ten years in the industry, there's exactly one time I was able to apply an "unusual" paradigm to solve a problem. GvR might curse me because I sort of made a Lisp-like DSL in Python but I stand by my design decision. The end-result was a small (~200LoC linted with black) DSL parser with some DB access and a lot of rules which actually tried to solve the problem. When I found something wrong I just had to reformulate a rule, not actually wrangle with loops, conditionals, and control flow in order to rewrite logic.
Honest negatives: I had to turn over the project to someone whose background is in EEE, not CS, and I had to explain Higher-Order Functions to them. Not ideal! Actually, in fairness, I bet most in that team would have a hard time figuring it out because, hey, I made a Lisp in Python! But I still stand by my decision; it was the best for the problem given the resources and time I had.
I commented on SWH a few months ago in HN too when another user, claiming background in linguistics, replied that it factors in your world view but not as huge a factor as other cultural considerations. That sounded reasonable.
So as of the last time I looked up SWH (a few months ago), my knowledge of its consensus is that:
- the weak form ("relativism") is true to a certain extent, and definitely not to the extent Whorf originally proposed.
- the strong form ("determinism") is largely debated and, personally, I think it's indefensible as a thesis.
> The strong version, or linguistic determinism, says that language determines thought and that linguistic categories limit and determine cognitive categories. This version is generally agreed to be false by modern linguists.
> The weak version says that linguistic categories and usage only influence thought and decisions. Research on weaker forms has produced positive empirical evidence for a relationship.
"language determines thought and that linguistic categories limit and determine cognitive categories. This version is generally agreed to be false by modern linguists"
The use of words like "determine" appears to make it look bogus. Whereas "influences" basically means the same thing (in practical terms, strongly influences is just as good as determines).
When I read the description, I understand "determines" as in, your thoughts internally work in terms of language processing, so their complexity is limited to the complexity of the language(s) you know. It sounds plausible, but also testable by checking if humans who never learned a language with complex grammar are capable of processing complex concepts.
As in absolutely limiting them, though? Like the Ancient Greeks, those who invented zero, didn't have it previously in their language, and couldn't do zero-based equations before either.
What do you think "disprove" means in this context though?
E.g. do you think they really settled anything, the way we can prove a math theorem or verify some physical theory to hold (as a predictive model) to the nth degree of precision?
>When I read the description, I understand "determines" as in, your thoughts internally work in terms of language processing, so their complexity is limited to the complexity of the language(s) you know.
That would mean that developing further language is impossible or even that developing the first languages would be impossible - as those nearly language-less peoples have no means to develop those more complex later concepts.
It's not like e.g. Europeans (let's take them as a model of "advanced" language) emerged in nature in fully formed ancient Greek, Latin, English, French, etc.
They started with the same or even more "constrained" languages (verbally and conceptually) as any primitive people, and - going back to our forefathers - even language-less.
What it means for every scientific hypothesis: it takes infinite amount of evidence to positively prove it, but it takes one counterexample to kill it. +/- the usual uncertainty caveats.
> That would mean that developing further language is impossible or even that developing the first languages would be impossible
Not necessarily. It would mean that you can't think about some concepts without first extending the language to support them. Languages still could be extended piece by piece.
Only in such soft sciences you can "prove" any number of competing hypothesis, and have fractions arguing about them for decades or centuries. And whether a "counter example" is valid and "kills" a theory, will be a matter of interpretation and endlessly debated as well.
>It would mean that you can't think about some concepts without first extending the language to support them. Languages still could be extended piece by piece.
If you can always extend a language to new concepts, the language is not limiting in the strong SWH sense. If it was it would be impossible to reach those concepts from a reduced set of concepts.
It took millennia for spontaneous generation to be definitively disproven.
So this might have been relatedto biology (or in other domains physics, chemistry, and so on), but the matter was debated in the soft sciences way, not in the way we do biology in the 20th century.
A lot of proto-linguistic (I say proto because linguistics want a real discipline yet) thought in the '50s-'60s did actually believe this. It was the informal basis for teaching lots of colonized peoples Western languages. It was the basis for the book 1984's Newspeak. It was also the hypothesis that motivated the creation of the Loglan (predecessor to the more well known, but still obscure, language Lojban) language. That you find this to be a strawman argument is a good testament to how widespread the knowledge of its disproval has become.
N.B. I also think in the pop programming language world (e.g. Twitter rants) that a lot of folks genuinely believe something similar to Strong Sapir-Whorf, at least as applied to programming languages.
I think it would be fairer to say that he was convinced that Whorfian effects were strong enough that there would be value in engineering a language to take advantage of them. But while value is somewhat in the eye of the beholder, I would say this hypothesis has not held up well either.
Even the weak version falls apart under scrutiny. Here's a nice example of this: https://cpb-us-w2.wpmucdn.com/web.sas.upenn.edu/dist/4/81/fi...
This is a good example of a case where there was a phenomenon (a language prominently using non-speaker-oriented direction terms, and correlations with real-world behaviour of speakers of that language) which seemed to support a weak version of SW, but it turns that seemingly supporting evidence was really just a by-product of the testing environment.
So this does present an interesting result, and one that (AFAICT---I am not a linguist by training) does run counter to the claims of SWH but, more importantly, the study is limited only to spatial reasoning. Surely there are more areas of thought than that?
Yes, this one study doesn't disprove every aspect of SWH, but it's a good example of how SWH claims, even in the weaker versions, tend to fare when subject to more rigorous investigation & analysis.
What area of thought do you think provides good evidence for (some version of) the SWH?
And your example, I think, also speaks to the point: you're able to reason in mathematical ways; being a speaker of English doesn't block this, even though English perhaps doesn't offer convenient vocabulary for this in all cases.
Though even in programming, there are e.g. non turing complete languages (e.g. simple regex languages), languages without certain high level facilities (making it much harder to organize your code in terms of certain concepts - analogous to organizing your thinking of the world around certain concepts -, which is a SWH-like effect), languages that can't express e.g. infinite loops and are certain to terminate (used e.g. for business logic, templating, etc), and so on.
Even if it all goes down to assembly, a language like Cobol might it much harder to write e.g. a 3D game. And that can be enough for games not be written in that language - it doesn't have to be impossible, just hard, which is enough to discourage most, aside from a few weirdos writing games in Cobol and compilers in bash.
You seem to respect everything labeled academic, but not all "areas of expertise" and not all "fields" are equally valid -- and surely not just because they are taught in universities, written about in "peer reviewed" journals, and so on.
Linguistics is closer to soft sciences than the opposite.
I assure you, I really don't.
> but not all "areas of expertise" and not all "fields" are equally valid -- and surely not just because they are taught in universities, written about in "peer reviewed" journals, and so on.
Fully agreed.
> Linguistics is closer to soft sciences than the opposite.
"Linguistics" spans a range of things from the humanities to the 'soft' sciences (in the sense that psychology and cognitive studies 'soft' sciences) to logic/maths.
On the one hand, most modern languages are powerful enough that you can build whatever you want on them. DSLs, constraint solving, pure FP, CPS, logic programming, whatever you like.
On the other hand, languages are subject to network effects. If you build an FP system on top of Python and I build one on top of Ruby, the resulting “FP frameworks” will be almost entirely incompatible. That makes it hard for people coming to either of our projects to “pick it up."
It’s great fun to implement your own library or framework that in turn implements something interesting, but everyone doing that leads to applications that are built on top of in-house home-grown greenspunned code bases.
In most production situations, it’s better to find the language and framework that already cater to the idiom you like, and come with a community of like-minded programmers who are familiar with how things work.
Network effects matter at scale, even if they’re irrelevant for a passion project.
The notion that any language can be coerced into supporting any programming idiom is true in theory, “But the difference between theory and practice is narrower in theory, than it is in practice.”
So I suggest there is still a lot of value in selecting the language/framework/library that was built to do the thing we want, right off the bat, and comes trailing a community and ecosystem around it.
The project was started for legal compliance so I don't want to go into specifics in case I make a slip one way or another. It did not run in the production cluster per se but was important nonetheless. (Actually, gods forbid it run in production!)
Very vaguely, it simplified structured data. A rule is recursively defined, base cases being simple rules that just returned primitive data. The original data could be an object with deep properties but most rules just simplified it to one with less fields.
I call it "Lisp-like" because after writing a handful of recursive rules, I had flashbacks of when I studied (actually) Scheme in uni. The tree-structure and meta-rules certainly gave that impression. I did not parse parens though, rather the rule syntax is JSON dictionaries. I hope this aided in readability---it should be self-explanatory what each rule did---and so my team did not find it too esoteric.
I don't remember the exact volume of data we processed but it was in the tens of GB, if not low hundreds. We parallelized it and had maybe 20 runners working concurrently. You can claim mapreduce (ha!) at this approach but honestly I think we could've parallelized it as easily if I did not go with this DSL-approach. :)
Countless systems have been designed around mutable data simply because they were written in languages where mutable data was the default. I personally worked on several Java systems where mutability was used but was trivial to refactor out, because every place an instance was modified, it was simple to construct a new instance instead. On a couple of them I had ownership of the codebase and was able to complete this refactoring with a reasonable amount of effort.
I also created a small codebase in Java with an immutable data model, only to pass it off to someone who immediately added setters for every field because they were "missing." And then of course he started to find uses for them.
I don't have a desire to work in Java again, but I think Java recognizing immutable record classes as a concept worthwhile of first class language support is a significant step forward for the industry.
Are you completely sure you didn’t just create a massive headache for the team that followed you? Now they have to understand something that looks like magic, or pay more money to hire a wizard.
I’ve seen a lot of “clever and elegant” solutions that become instant legacy when the original genius leaves, and a lot of “boring and repetitive” solutions that stand the rest of time.
Complete certainty of how it turned out is a bit too much to ask. For every story like mine, there's an equivalent story of a legacy system in boring Java/C that no one wants to touch, works like magic, etc.
FWIW, I wrote a pretty extensive documentation of the project before I left, including recommendations for improvement[1]. Again, it's foolish to claim complete certainty but the nature of the project was such that they are unlikely to need to rewrite anything on top of the engine. Any modifications and additions moving forward would be in the declared rules, not in any of the logic. And for that, they have hundreds of examples from me to copy from.
And if they do need to touch the "parser", at ~200LoC the complexity is certainly far below, say, a Spring J2EE code base. It's all in one file too, that shouldn't be too bad.
[1] If I may use HN as a small soapbox, actually, months later, I figured out a further small addition to that project that would've made working with it far easier. If any of my previous team recognizes this and this project is still in use, feel free to reach out to me for one last brilliant idea. :)
> It's all in one file too, that shouldn't be too bad.
Fewer files is not necessarily better.
The guiding principle in my mind is reduction of entropy. If splitting one file into multiple files makes sense in the brains of the beholders it is a good thing.
> [Lisp macros] by their nature would be hard to implement properly in a language without turning it into a dialect of Lisp
Other candidates that come to mind (because I recently learned about them) are effect systems, like in Koka.
The rest of lisp is pretty easy if you've done any sort of functional programming, and view the code through a lens that translates (foo ...) to foo(...)
The main difference is: mixins are compile-time only, you can't add them to a running system like in Lisp.
In practice, it is a much smaller difference than it looks like.
A better analogy for new materials is new techniques, like machine learning and higher order abstractions over it, or new algorithms. But those are independent of languages.
I'd still like, as a learning exercise, to build some semi-practical small applications in Prolog, but I feel like I'd need an expert mentor to guide me.
FWIW I do actually agree w/pg about macros; I use them day-to-day for more mundane things than his impressive fully-bottom-up approach to programming. But each time I do use them, I feel like "thank heavens I don't have to reach for some weird external codegen tool to do this".
Don't get me wrong, I think homoiconicity and macros are really neat, I can get behind the aesthetic appreciation of different programming languages and syntaxes for doing different things. But people seem to want to convince me that there is some mystical mind expanding power to it, rather than just aesthetic nicety, and that's what I find unconvincing.
Then I got my first job, where they really only cared that I was good in python. Cranked out a bunch of python services, and got good at cloud stuff along the way. Decided FP and that kind of thing was "school stuff."
A few years later I found myself working at another shop doing backend Go/Kotlin. We were increasingly writing a lot of streaming stuff on Kafka, and could tell we weren't really "doing it right." Just gut feel that something was not as good as it could be. So we brought in a senior eng with a bunch of Kafka experience, who just sort of coincidentally had been working in Scala during that experience. We onboarded him onto Kotlin, but the code pretty quickly picked up a Scala style and flavor.
After about a year I couldn't shake the feeling of how perfectly this kind of functional thinking mapped to stream processing services. Especially when you're working on a framework like KStreams or Flink, being able to describe your transformations with these pure non-nullable functions was just perfect for the domain. When you squint a lot of the Option/Either monad stuff really helps you think about your programs as a linear flow of data, which is exactly what stream processing is.
When you look at a lot of the Java api's in the space of big data, they look functional. AFAIK thats not because a bunch of Java devs decided to use all the cutting edge Java functional features. Its because the stuff people were building in Scala just fit the domain so much better. So the Java ecosystem took those ideas and everyone is for the better.
At no point did I develop a superpower. Now I'm back to writing Python stuff in a new domain, and those insights aren't so valuable day to day. I try to make my code a bit more linear and functional, but can increasingly tell its not the practical choice. But if somebody needed me to write a high throughput reliable stream processing thing, I would immediately jump back to those patterns.
This can expand to your tastes or everyday life by not wanting to learn X "just for the fun".
> 99.5% of programming consists of gluing together calls to library functions. All popular languages are equally good at this.
In my experience, there are tons of programmers who balk at anything more complicated than a for loop, to the point where they won't even read the surrounding documentation of a short block of code to try and understand what it does. That's the 99.5% of programming that pg is talking about. I'd guess that some 80% of programmers spend their entire lives in this regime. When algorithm/language researchers talk to software engineers about what programming is, sparks fly. Like the parable of the blind people examining an elephant, we're talking about the same thing, but our experiences with it are too different to reconcile.
At least mention prolog, forth, SQL or the like that are truly different.
Lisp has its macros.
Elm has its type system.
Go, Elixir, and JS have rather idiosyncratic concurrency models.
Rust has its borrow checker.
Perl seems to be SO special that even proficient Perl devs aren't able to write performant code in it. /s
Sure, Prolog and SQL are much more different than the rest. But I didn't want to strawman here.
(It is, however, difficult to disentangle some features that Rust’s ownership model requires but which aren’t necessarily a part of that ownership model: for example, algebraic data types are essential and expression orientation extremely desirable, but there are other languages which have those features without the rest of the ownership model.)
People often think memory safety without garbage collection is Rust’s shtick, but it’s actually the ownership model that’s Rust’s defining feature, underpinning everything and making that possible.
And the idea that a language can be designed around one data structure is elegant. Of course you get that in Lisp and Smalltalk as well. But having all those primitives that work over array structures is incredibly expressive.
I believe the array podcasters when they claim array languages can be several times more productive than other languages, at least for domains that fit well with arrays.
This is a bad analogy. Programming languages all get compiled to the same machine codes. The human-readable version of that code doesn't actually matter very much, if at all, after compilation. Programming languages are tools for humans: they're either expressive enough or they're not.
A better analogy would be rocket engineers saying the CAD tool you choose to design your rocket in doesn't matter—and that has some merit.
1) Programming language quality is a non-factor in business success.
2) People like PG are wrong about what makes one language superior to another.
By the by, I have never seen a non-programmer executive demand a “bad” language be used for a startup. What I tend to see is grizzled ex-programmers ignoring the trends and picking the reliable and “boring” choice over the complaints of some IC’s. This is less a “clueless business” issue and more an acknowledgment of an unpopular truth: picking the boring language that lets the business focus on its core domain is better than picking something that excites the most vocal engineers.
If it pleases you, feel free to learn whatever. It’s why I know Rust and Haskell personally. But as career advice I am much more dubious, even though I once agreed with you.
So while I agree about writing prose, it's also true that learning lisp made me a better documenter.
That assumes that the software business is efficient in the market sense, which my experience suggests is very much not true. On the contrary, I've found that success hinges primarily on the ability to find and execute good business deals (which is the work of executive team and/or sales&marketing). Software side has so much slack in it that it can (and often does) do extremely bad job at making the actual thing and still succeed.
> I have never seen a non-programmer executive demand a “bad” language be used for a startup.
I've worked for one many years ago. I was actually promised the free reign over technology stack for a project, until the CEO decided to override me and order me to use a very particular and shitty combination of technologies. He was very up-front with me about his reasoning: he said, "yeah, I could let you write this in something more reasonable, say Ruby or Python, but programmers for those languages are expensive, PHP programmers are much cheaper, and I'm going to be hiring more people to help you".
(I run the project solo for the next year, because he eventually decided not to hire anyone.)
> picking the boring language that lets the business focus on its core domain is better than picking something that excites the most vocal engineers.
Be careful not to mix up topics here. What usually "excites the most vocal engineers" these days is the things they can use to pad their CVs. This usually goes against what the management wants, which is whatever stack they've read about on HBR or whatever one is cheapest to hire people for. Neither of those tend to have anything to do with powerful and advanced tooling.
I actually covered this in literally the next line.
> 1) Programming language quality is a non-factor in business success.
What TeMPOral wrote (below) is is not quite the same in my view. First, they elaborate more. Second, there is a different argument:
> Software side has so much slack in it that it can (and often does) do extremely bad job at making the actual thing and still succeed.
Namely: it is NOT the case that software does not have a business effect. Rather, other aspects of the business have to work extra hard to compensate.
To take a systems point of view: two systems can have the same output over some time period but very different internal efficiencies.
It sounds like the exec made the right choice. PHP is a wildly successful language that powers an obscene portion of the Internet web apps.
That’s not a “bad” language, it’s just an annoying one for engineers to work with. A bad language would be one with so little adoption and such a steep learning curve that hiring new software engineers is a year-long path to expanded productivity.
Similarly, having been forced to learn Prolog in university, I now have a completely different view of solving problems. And while I'll surely never touch Prolog again, this new perspective allows me to write much better code in some cases, despite working in a completely different language.
In the end the most effective people around me are capable of designing a system and breaking it down effectively for other engineers to build. Most of them have no idea what things like Monads and Functors are, and they’re no less effective for it.
I think it's possible you are falling prey to the either/or fallacy.
This ideology is why I believe so strongly in the power of SQL (and similarly-expressive functional/DSLs). They are effectively limitless in their ability to describe problem spaces. In SQL, joining 30 different dimensions of data is a matter of the same # of lines of SQL. Doing such a thing in any imperative language would be a medium nightmare at best. Even with LINQ at your disposal, this is not fun.
I actually did not realize it was possible to construct a useful piece of business software that exclusively uses SQL for its domain logic until ~18 months ago. Turns out the trick is to go all-in and just make sure 100% of the relevant domain state is represented in the schema. You also have to understand why database normalization is important if you go down this road. The only reason you would fail to model something properly is lack of imagination, experience and/or discipline. These tools are limitless.
There is also the benefit to the business. You can print off all of the tables as excel spreadsheets and review them with non-technical stakeholders. I find this much preferable to explaining how procedural code works to non-wizards.
So, I would say - Weird languages - Absolutely yes.
But, also go check out some of these new tricks you can teach to old mainstream things. If you want to play around with these ideas, getting SQL into your application is absolutely trivial with SQLite these days. We are using it in production for this purpose right now. My project managers are more familiar with the product's schema than I am at this point, because they are authoring BL in SQL today. This is where you really want to be, IMO.
I like this way of business development: Prototype some SQL, place it in Metabase, get feedback, update the SQL, when they are happy, make it to an Update that solve their need automatically.
And it gets way more powerful when you learn Macros in PL/SQL or PostgreSQL so that you can dynamically change the SQL based on results. One example is to build the Metabase functionality into the business application do that you can easily add/change SQL, and the resulting values automatically is clickable if the data comes from the right tables/views.
Rollback is always difficult on live systems, especially when the data propagate to integrations. I try to always have 'first created' and 'last changed' time stamp, and first created and last changed user. If the job is automatic the “user” should be traceable back to the individual job, not some generic system user.
Sections 5.1.1 and 5.1.2 in my side project demonstrate what I mean by "joins are values": https://docs.racket-lang.org/plisqin/Refactoring_Recipes.htm...
Language A has features 1, 2, and 3. Language B has features 1, 2, and 4.
Language A user thinks they're superior because they have feature 3, which B lacks. B's feature 4 they don't understand, but they've never needed it so it's useless.
Language B user thinks the opposite, of course.
(this was much better explained in an article that graced HN a while back, and I can't find anymore)
The language of the System - Rich Hickey https://www.youtube.com/watch?v=ROor6_NGIWU
But I'll probably never use a functional language in production.
The Rust borrowchecker is an awesome idea, I don't quite like Rust and don't want to use it, but I'm excited for the concept to be introduced into established or newer languages - and it helped me reason about my own programs a little bit.
Even beyond personal curiosity, that was time well spent.
It also seems plainly mistaken. We still see a steady stream of security vulnerabilities relating to imperfect use of unsafe languages. Safer languages make those sorts of errors categorically impossible. Even if your position is sure but these languages aren't always practical, that's still a concession that further work is needed on safe languages.
That said, some languages are more ergonomic than others, especially when you compare them against particular domains. But IMO that's a problem that transcends language design. All tools, not just a language, should be fit to the domain. And in general you can build APIs and libraries that can reasonably fit any language to a domain.
Edit: slight simplification.
And in that sense, I think the value of 'weird' languages is that they can package so many different mutually-reenforcing design decisions together that they literally can have the 'think different' effect described in the essay.
Taking one example that's kinda-weird, kinda-mainstream -- a 'language' like Erlang includes way more than a Prolog-inspired syntax for a simple functional language, and when you look closely at it, it's not simple to just recreate by pulling in features ad-hoc; the language, its stdlib, and its VM complement each other and have informed each others' design.
Probably the feature that most comes to mind when we think of Erlang is that it 'goes all-in' on message passing in a functional language. 'All in' meaning that they've made decisions like: being willing to copy a lot if needed, little details like keeping a count of reductions, big things like pulling in concerns that are often left to the OS when working in other languages like process scheduling, preemption, and supervision, and providing a toolkit out of the box that supports building full systems via Nodes with Applications with trees of Supervisors of Processes (all of which are effectively language concepts now), etc. Well, every one of those decisions is eyebrow-raising to somebody or a lot of somebodies. But if you do go all-in on that combo of capabilities, what's it like?
A good engineering team with a lot of money can fit another language and another VM to the same kinds of domains that Erlang is a fit for. (Not that you can get people to agree what those domains are, even within Ericcson apparently). But their solution will necessarily look very different, and the difference between what's easy/hard in their approach vs. an erlang-ish approach is what the weird language can teach us.
When programming, I am thinking in terms of data, what processing the data needs, how will you structure the data in memory/disk, what IO/integration is required etc. This thinking has nothing to do with what the programming language allows/makes you to think. If your programming language is supposed to dominate your thinking about programs then you are just constraining yourself to a specific way of solving the problem.
Linux, llvm, Android, iOS, git, cPython, numpy, etc, etc. From operating systems, to compilers, to scientific computing, to literal rocket control systems, it’s mostly boring C/C++/obj-C/Java.
And facebook itself is historically PHP (where they had to fork the runtime even), but they use plenty of other boring tech (eg. Java), as well as some exotic ones, notably haskell for some query pre-optimization (afaik)
I think this heuristic fails, with my poster-child counter-example being Nock and Hoon, the languages used to implement Urbit. Those languages are unquestionably weird, and I think their median user is pretty smart -- you have to be pretty smart to use those languages because they are intentionally designed to be difficult to use. But I don't see any merit whatsoever in Nock and Hoon, at least not with respect to giving you any leverage in programming. C++ is another example. It's not considered weird, but I think it would be if it were not so widely used. Again, to use C++ you have to be smart because you have to know the locations of a zillion and one hidden land mines. But the appeal of these languages is not the leverage they give you, it's the barrier to entry. Once you invest the time to learn them you become a member of an exclusive club. In the case of C++ at least, being a member of the club gives you leverage and opens up employment opportunities. But that doesn't make C++ a good language.
TFA is about weird languages, not good (or bad) languages. It's about aspects of a programming language that are not part of the intersection of "95% of all programming languages".
> if you want to expand your concept of what programming can be, one way to do it is by learning weird languages.
Implicit in this advice is that programming is actually a useful endeavor and not just a puzzle like sudoku that you engage in mainly to pass the time. And I'm saying that the weird-language heuristic is as likely to lead you to something sudoku-like, where you face a lot of interesting intellectual challenges, but at the end of the day you don't actually get any leverage towards doing anything that is actually useful, as it is to lead you to something with actual utility.
But in general programming is viewed as something one does in order to achieve some practical purpose, not just to while away the hours. And with respect to that goal, "seek out weird languages" seems like a poor heuristic to me.
C and C++ were weird at the beginning and their median user was smart. They just become so mainstream that, for many, programming is programming in C-like languages. The majority of developers nowadays won't find new concepts, new ways of thinking by picking up C or C++, they're likely to have met those concepts already.
If the majority of developers picked up Lisp or Haskell, they would likely discover new ways of thinking.
This is changing, because popular features, abstractions and ways of thinking are trickling down from Lisp and Haskell to more mainstream languages.
That's right, and I never said they were equivalent. But I think being smart is a prerequisite to navigating C++.
> If the majority of developers picked up Lisp or Haskell, they would likely discover new ways of thinking.
Yes, I agree. But the problem is that the weird-language-with-smart-users heuristic is as likely to lead one to Nock and Hoon as it is to Lisp and Haskell, and I don't believe that you will discover new ways of thinking in the former. What you will discover (though it may take you a lifetime to realize this) is complex obfuscations of simple ideas that were discovered long ago.
Now, when I first modified Nix I saw some surprisingly readable C++, but this was all performance non-critical code where C++ wasn't needed. In situations where the use of C++ is actually justified, the weirdest is much more pronounced, and the cognative burden much worse.
Definitely agree Urbit is a trap, though.
Maybe, but at a minimum pointer arithmetic would appear to be an exception.
> What can you say in this language that would be impossibly inconvenient to say in others?
The first experience that comes to mind for me was XSLT. Early on in my technical writing career, I needed to convert the Doxygen output of a C library to a more barebones HTML output (so that the reference docs would have the same branding as the rest of our site). I used XSLT to extract only the bits of HTML that I needed and I transformed them into nice, straightforward, semantic HTML. It took me a while to wrap my head around XSLT's flow but I was amazed to find that a task that would have taken probably 50-100 lines of looping and condition checking could be accomplished in literally 1-3 lines of XSLT.
Would love to hear other's experiences along these lines (i.e. concrete examples).
Pandas/Numpy is close but R's macro/custom operators make everything much more seamless.
R is definitely weird language, not as useful for general purpose computing, and has far less marketshare. But that's kinda the point of pg's essay.
XSLT is basically SQL for structured text.
That is what XQuery is for.
Then you try to e.g. format a date or something.
As long as you stick to what it is good at and relegate everything else to another layer, it's fine (it could certainly be a lot better, though)
XSLT doesn't do itself many favors, either, because the model depends on the execution of a few "hidden" default templates that drive the tree processing. If you don't know those are there or what they're doing -- or worse, if you don't know they're there and you accidentally override them -- then it's really hard to understand what the heck is going on. It just looks like poorly-defined magic.
Plus, you need to understand XPath to write effective templates and most people don't understand it as well as they think they do. It's deceptively powerful.
Finally, add on the fact that XSLT programs are themselves valid XML -- who the heck wants to write code in XML! -- and it really never had a chance to be understood, much less liked.
But many of its ideas were good!
For example, consider logging libraries. It's useful to have statements like log.debug(someSlowFunction()) in your code. In LISPs it's really easy to create a (debug) macro that generates empty code when you don't want debug logging turned on. In other languages, you have to wrap the arguments in a function to avoid extra runtime costs, and even then you can't avoid the "if debug" conditional at runtime. All those anonymous function wraps add clutter, and that clutter accumulates. There are many other cases where having advanced language features greatly helps gluing together libraries.
Another aspect is the tooling. When I am considering a new library, I like to try it out in the REPL. In Clojure I can quickly start calling library functions, and use the (time) macro to get a sense of how long they take to evaluate. Not all popular languages are amenable to this kind of REPL-driven experimentation with libraries.
Not only does the language impact how you use libraries, but it also impacts what libraries may exist. Some libraries are simply not possible to write in less powerful programming languages. In LISP this would include any library that uses a macro. For example, Clojure was able to introduce core.async as a library, providing an async facility similar to what golang offers. But in most languages you wouldn't be able to implement that as a library.
Another major example is reagent vs. react. The concise hiccup representation supported by Reagent is only possible because of design decisions that went into Clojure. JavaScript users are stuck with JSX, which is less concise, and in my opinion far less good.
Another issue that arises when using libraries is whether or not the language has a static type system. Without getting into the age-old flamewar about static vs dynamic typing, I'll just note that popular languages differ in this dimension, and this has a big impact on what it's like to glue together libraries.
So overall, I think this essay undersells the benefits of LISP. Even if you spend all day gluing together libraries, LISP makes that much better by improving how you can call libraries, how you can quickly experiment with them, and even what kinds of libraries can exist.
Another great example of this is feature flags -- adding/removing an entire chunk of code in most languages tends to be limited to (for example) the inside of a function.
(when-feature :something
(def some-constant ...)
(defn some-function [...]))
> JavaScript users are stuck with JSX, which is less concise, and in my opinion far less good.
And a key aspect to it as well is that JSX is a fully-custom extension, not something that can be implemented in JS as a library -- whereas Hiccup is 'just another library' allowing for fast iteration and experimentation by the community.
I've used environmental variables to determine which modules extend a class. This allowed me to test the same code against two different apis while we were migrating vendors. While this is done at "compile time", it could have easily been done during run time.
One nightmare of a system I worked on had the initializer of a class conditionally load modules to completely redefine the class. I called it reverse inheritance because it did the same thing as inheritance, without any of the readability or simplicity. Just because you can do a thing doesn't mean you should do a thing. XD
But I think Ruby is the exception that proves the rule. It allows absolutely crazy things and probably deserves the label of being a weird language. It's just wrapped in the trappings of a non-weird language.
A constant false value used in an if statement will trivially be optimized away in case of any non-toy JIT compiler.
The javadoc for slf4j's Logger.debug implies that the JIT cannot be relied on for this:
http://slf4j.org/apidocs/org/slf4j/Logger.html#debug(java.la......)
"This form avoids superfluous string concatenation when the logger is disabled for the DEBUG level"
I think that means if you call Logger.debug("a"+"b"), the string concatenation happens even if DEBUG logging is disabled. But maybe JITs have improved since that javadoc was written, or the author was not aware of how smart JITs are.
I would be curious to know if there is a JIT expert who could weigh in on this question.
It's one of the simplest and most commonly implemented peephole optimisations - perl has handled it for as long as I can remember, I'm pretty sure cpython does, etc. etc.
> For example, consider logging libraries. It's useful to have statements like log.debug(someSlowFunction()) in your code.
Java/C# have good debuggers and thus you can set breakpoints.
You often want to have a debug printing in final release build anyway, activated through a --verbose option for example.
Thus I think this is less useful than it seems.
> Another aspect is the tooling. When I am considering a new library, I like to try it out in the REPL. In Clojure I can quickly start calling library functions, and use the (time) macro to get a sense of how long they take to evaluate. Not all popular languages are amenable to this kind of REPL-driven experimentation with libraries.
In python REPL you can do a reasonable level of experimentation with libraries. You can't do weird stuff like runtime monkey patching but that's not always considered a good thing.
In java also, there's jshell. (importing maven libs is little roundabout, but there are packages for that, and in return you get a solid ecosystem with static typing and great compilers / runtime.
> Another major example is reagent vs. react. The concise hiccup representation supported by Reagent is only possible because of design decisions that went into Clojure. JavaScript users are stuck with JSX, which is less concise, and in my opinion far less good.
It's a matter of opinion. In my opinion, jetpack compose or flutter (admittedly, fringe language but can be learnt by anyone with java or c# experience), are better way of doing UI than jsx / fxml etc..
> Even if you spend all day gluing together libraries, LISP makes that much better by improving how you can call libraries, how you can quickly experiment with them, and even what kinds of libraries can exist.
I think benefits of macros is way overblown, and higher order functions with ocassionally well documented APIs that use compile-time or run-time reflection are good enough. Macros are not worth the readability and compile time sacrifices.
Overall, my point is, blub paradox isn't one dimensional. I place more importance to tooling and libraries than language itself, as long as language isn't horrible.
A common complaint at Google was "all I do is copy one protobuf to another". And I think that sums up programming -- that really is all we think we do. For example, a triple-A game just reads a stream of keyboard events, mouse events, and network packets, and emits streams of video frames, sound samples, and network packets. The rest is just details!
That is obviously not true, of course. We find the interfacing the most tedious, annoying, and painful part and overestimate how much time we spend doing it. Interfacing with anything is painful -- I had the idea for this comment in an instant, but I'm spending 5 minutes thinking of words and sentences, figuring out what order they should be in, and commanding my fingers to press keys on my keyboard to get it to you. But it would be incorrect to say that "99.5% of human cognition is convincing your fingers to press keys", even if that's a strict blocker to disseminating your idea on the Internet.
What that means is that tools can help us eliminate the tedious parts. The fact that tedious parts haven't gone away just means that there is more work to do. When the tools start doing what we need, you'll stop seeing the complaint that all programming is is moving data from one library to another. You'll start seeing the complaint "all I do is stare at the screen figuring out what to build."
99.9% of programming is a lot more of just plugging together library and API calls. We plug them together to user interfaces which is the apex of the age we live in: technology, empowering people to change the world in exciting new ways (as hackneyed as that sounds). It's just true. Our age involves a lot of globalization (in the Peter Thiel sense), but it's still very squarely part of a technology revolution.
Viaweb might've been a way to just make money for the most part. But that's not what impactful SaaS or other software is about. What the APIs / library calls are being connected to is of utmost importance.
And similarly, the ideas that Shakespeare came up with - the hundreds of ideas, that help us focus on what's most important to live a meaningful life, those are also helpful. And yes the language plays a role.
That said there is a grain of truth to this parent comment and PG's post. But there's also a lot missing. Both words and C++/Java-esque languages are super basic. But with enough of a fixed idea in mind, over time, they're good enough for getting at some really important things. And that is what matters.
First, my own career was not as pathologically skewed as this. I did mostly glued together calls to library functions, but I have done a fair share of algorithmic work, such as parsing and image processing. (And cryptography, though that was mostly done on my free time so I’m not sure that counts). Maybe web dev is like that, but I didn’t do web dev.
Second, this is a huge problem. This means that very few of us actually know how to program. And the problem feeds on itself: if one doesn’t know how to program, it will be easier, more tempting, to take on some significant dependency instead of just implementing the part you need, while tailoring it to your use case. Eventually knowledge is lost.
There're thousands teams all over the world working on various projects which require people to program. At the very least, someone needs to write and maintain these libraries. Also there're interesting domains like game development, CAD/CAM/CAE, HPC, embedded. Knowledge is transferred within such teams.
Code reuse is good, but we should not forget that its costs are not nil.
So for engineering efforts that require large amounts of people to contribute, a more restrictive, less flexible language can yield better overall productivity by expanding the pool of potential contributors and reducing friction arising from pointless differences and ‘smart’ programming (like macros).
In other words, removing ‘smart’ programming features like macros probably turns out to be (weirdly) smart in the real world.
At the same time I enjoy tinkering with different new languages every once in a while and I think that's what the article talks about. There's a place for experimenting and trying out new things, that doesn't mean that you need to rewrite everything in Haskell.
Eg. on my current CRUD app, we use Spring’s aspects to implement some permission checking before specific HTTP requests. The alternative - copying a function call to each place - would be much more bug prone, a change in one will not be propagated, etc.
While it may not be the best example, there are many similar cases where I feel that less abstraction would just make it unmaintainable.
mux.Handle("/", middlewareOne(middlewareTwo(finalHandler)))
https://www.alexedwards.net/blog/making-and-using-middleware
My hypothesis is that languages that are good at making the individual feel smart have proven to be less effective at allowing a large group of programmers to collaborate.
I am offering a counterpoint that instead, some ‘smart’ engineers have concluded that boring programming languages are the way to get big things done.
Addressing a skill gap technically seems a bit short sighted compared to addressing it culturally. Especially because most weird language features don't actually require you to be smart to understand them; they're just unfamiliar.
It quite simply was too much for some guys to get their heads around. It probably wasn't the best way to use an exception either, but occasionally when I'm working with .NET I really wish I could catch some of those, re-inject the expected bit of data and have my program continue it's merry way.
One of the modes of working in Smalltalk is to deliberately write something half finished, so when the #doesNotUnderstand gets thrown, you can edit the code during while debugging, accept it, and continue the program. If it takes a long time to reproduce a bug in a C# program, you're stuck with going through all the motions to get back to where the bug exists, but in Smalltalk you could simply fill in the bit of broken code, continue and it would work from then on.
Again - hard to change the habits of a lifetime and I'm not sure that as a long term development environment it's very healthy (you might fix an instance of the bug, but did you fix all the possible combinations of it?)
Still, when you're in a Smalltalk groove it feels like walking a high wire and performing magic.
There are "business as usual" problems, so gluing libraries, CRUD's. Where having junior level code is by far most efficient because you probably have to face multiple issues. Ones like employee rotation, lots of business requests where it is by far better to have code duplication and some tweaks in copy pasted code so it stays simple and don't introduce wrong abstractions. Even generic types as in C# I would consider mostly bad idea to use for those.
Other type would be libraries or frameworks where you definitely cannot afford "copy/paste/change". I would hope that people working on libraries or frameworks stick to what they write for at least 5 years and hope they could be life long supporters. For those people and types of projects you want some smart features like generic types so that they can simply do their job. I cannot think of a framework that would not use quite complex ideas to achieve what it should be.
Of course there are many more like hardware programming, operating systems and the ones that PG pointed as 0.5% of developers work like really hard/weird problems one cannot solve by simply making CRUD.
I have pointed only at things that are in my line of work. Unfortunately there is a lot of people who should be delivering things as in CRUD/glue/junior so the first category. Where what THEY think they should be doing is some kind of framework/libraries work. Because they think that it will make business "future proof" and "that is what professional developers should do", while wasting resources.
Of course doing CRUD/glue in an easy way that can be understood by juniors and fast is important and highly valuable in itself. But I think a lot of people don't understand value of it.
And it is written in C, which I assume is not one of PG’s blessed ‘weird’ languages that supposedly superior programmers are using.
unsigned short *reg = (unsigned short*)0xFFFF1014;
*reg = 0x1027;
Sure, you can crash your program if there isn't the hardware you expect at 0xFFFF1014. Sure, you can use this exact approach to trash memory anywhere in your program image. But it lets you easily do things that are not easily done in most other languages.
By that definition all languages are ‘weird’ including Java and have uniquely expressible koans within them.
Maybe it boils down to ‘use the right tool for the job’, ie, the tool that lends itself best to the job at hand.
I am not writing about one or the other being better, so I would like to see more context from your point of view.
I enumerated problems as I understand those and tried to match proper tools - while thinking that as a lot of businesses pay for CRUD work, they think it is valuable as in monetary value not in some absolute terms of what we can achieve with software development. I don't think solving hard problems that are "0.5%" of development work done in some esoteric language is greatest achievement or best in absolute terms.
1- I have never seen a language that made me wow. Like, to me, this whole concept of making a digital logic architecture do a task should be "solved" in a systematic/mathematical sense. Yet, whenever I read on criticisms of languages, people always talk about stuff like "Go doesn't have generators", "Rust macro is a mistake", "I don't like python's async story", "C's way of things is outdated" etc...
As an analogy, this whole discussion of PLs in CS circles is like complaining about whether we should use `dot` notation or `d/d(t)` notation or whether we should use `t` or `x` in solving a differential equation. Like, we are not solving the actual DE, but complaining about issues irrelevant to the big picture.
2- I have a feeling that https://www.red-lang.org/ is something completely new and innovative.
3- I think concepts like formal-proofs and RTOSes are under-rated.
Until you start manipulating the 'dt's on their own which wouldn't occur to you if you were using the dot notation.
(And then some people tell you that it's illegal, or maybe okay but only if you've taken real analysis... it gets confusing)
The Haskell version reveals more of a deficiency in the C standard library than the language.
The first Prolog program I ran just blew my mind - sort of "But how did it do that!?" I understood what I wrote down, and how that explained the problem, but a mental model if how that was used by the runtime to solve the problem needed a while to develop.
Similarly Smalltalk was like: "Everything is an object?". I mean the language doesn't have an "if" statement! You send a code block as a message to a boolean value with an "ifTrue" method.
Not that I know these languages to any significant depth, just dabbling.
To me there are a few oddball languages that are simple, have one abstraction of abstraction and are quite weird, but have a stack record of "heroic deeds" by their practitioners.
I mean C#, Java, C, C++, Go are quite similar if you squint.
But Prolog, Forth, Smalltalk, J, Lisp, etc. are definitely something else that stretches your mind a bit in different directions.
And I think that this is a valid point.
It's worth spending a bit of time reading about and playing with these languages. Not because they are useful (although they might well be), but because they will give you a few different ways of thinking about problems that will extend your mental reach.
Incidentally, I don't think that this is how they got there, but this could have come from taking the lambda calculus extremely seriously: in the classical encoding, `true = fst` and `false = snd` (i.e., `true x y = x` and `false x y = y`; or, pointlessly, `true = const` and `false = true id`), so that `if` is just `id`, in the sense that `if true = true` (meaning `if true x y = true x y = x`) and `if false = false` (meaning `if false x y = false x y = y`).
https://en.wikipedia.org/wiki/Lambda_calculus#Logic_and_pred...
The fact that makes all the difference is that informal notations in math, science, linguistics and other fields are interpreted by human brains, human brains are flexible at filling the gaps and deriving most reasonable consequences from their incomplete descriptions. Symbolic notations outside of formal logic and programming are mostly an annotation DSL to augment humans communicating with each other.
In contrast, a programming language imposes itself upon you in a way that those notations don't. You have to describe a lot of things in it. Underspecification leaves too much to the machine, overspecification tortures the poor brains reading after you. The optimal boundary differs depending on the thing being described, might not always exist and sometimes not always that satisfactory (still pisses off the humans or the machines in some parts or another of the program).
Making different programs in different languages and making them somehow all cooperate to describe the same problem opens a whole new can of worms that's just as hard to solve as the original problem. It's nowhere near as hard to mix-and-match mathematical notations (e.g. Lagrangian formulation of classical mechanics where Newtonian dot notation is used to denote time derivatives and Leibnizian fraction notation is used to denote coordinate derivatives). If you're stuck, you can always invent a new notation or whip up a diagram, describe how they work on-the-fly in some shared meta-language, then use it right away anywhere and everywhere you want, including in conjunction with old notations.
So it basically boils down to the capabilities of the underlying machine. The human brain is one hell of a machine, you have a lot of leeway. But programming languages are primarily ways to communicate with a very different type of machine, and as if that wasn't enough they _also_ still have to support wet brains, and as if that wasn't enough they are extremely rigid and heavyweight compared to human notations, i.e. much more difficult and time-consuming to design and implement and spread (when they are new) and de-spread (when they are obsolete and there are better alternatives).
For example functional languages and their monadic construct supports capturing various interesting aspects. This allows for really useful feature such as capturing concurrency behaviour (Future/Promise) and possibility of returning errors (Try), at compile time.
Then there's Rust where memory ownership can be reasoned about at compile time.
Languages such as Idris allows to declare a type that is dependent on the value (eg., declare that this function "x() can only return odd numbers?)
I also happen to work in a company where we try to bring type-safety to the databse world with our product TypeDB.
I think this trend is very good to see - stronger type-system = safer code. In a way the stronger type system a programming language has, the more it can validate at compile time. And therefore, we can say that such language somewhat provides what formal proof / formal verification systems ought to provide, but in a practical setting.
Math does have a lot of notations that express the same, but are used for different purpose, where some of those are better suited for solving some task than others. One of the simpler examples is polar vs cartesian coordinates. Where polar coordinates is expressed by angels and distance from a zero point. While cartesian we use the distance on axis from the zero point. Both can express all the same points, but some problems are easier to solve in one of those and some are easier to solve in the other. Coordinate system is definitely not the only place where you have this in math, and it is probably one of the simplest examples.
So math, and all it's different ways of being expressed is more of an example why we need different languages, and that programming languages matter. Than a counterexample of why languages shouldn't matter, and that we are fine with any one language.
This is surely the only chance I'll ever have to say "non angeli, sed angli", and by God I'm going to use it.
A majority of Haskell's innovations have been in the space of placing interesting new kinds of constraints on code
Rust's big innovation is not allowing you to make memory errors (in designated sections of your code)
Go's headlining feature is not allowing people to write code that's extremely abstracted, which helps readability
And this isn't just for avoiding mistakes: code that's very regular can be more easily processed by a machine to make it more efficient
So looking at it through the lens of syntax is missing a lot of the picture. You're right that syntax is rarely important. At the end of the day the differences between code and notation are: 1) code often grows to enormous sizes where managing it becomes a challenge, and 2) code is processed by a machine
I’d actually say Rust’s ownership model is its defining feature and big innovation; memory safety without garbage collection was the original goal of the project, but that has arguably actually ended up being more of a side-effect of the ownership model than the actual artefact, from the innovations perspective. Several other languages have been adopting significant ideas from Rust’s ownership model without actually needing it for memory safety.
(Also, some factual clarifications on the wording you used: Rust doesn’t allow you to make memory errors except in areas marked with the “unsafe” keyword—you’re marking small dangerous zones, rather than marking safe zones. But scripting languages are normally memory-safe too, it’s the absence of any garbage collector that makes Rust special there.)
Strongly typed SQL/XML/CSV/JSON without boilerplate is a massive leap forward, and it's a shame that it hasn't caught on.
Idris has them (but it's Idris) https://docs.idris-lang.org/en/latest/guides/type-providers-...
This SO answer states that Java has something similar called "Type Manifold" https://stackoverflow.com/a/53037742
I only have F# experience so I can't comment about the ergonomics of the above. I love, love, love the JSON/Regex type providers. I'm trying to move away from SQL (to event sourcing) so haven't used that type provider yet.
Main point:
> Theory, Kaiser suggests, is ultimately less important to theorists than the tools that mediate their calculations.
To me, programming languages are almost like Feynman diagrams or chemical notation in that they define how we think or what we think is 'real' in programming. In that sense, we should absolutely care about the fine-grained characteristics of the tools. After all, doesn't a chef prefer a sharp knife to a blunt one?
This isn't to say that the choice of language doesn't matter -- it does. If your team knows C then you probably shouldn't try to start a new project in Java without some ramp-up/learning time. If you're working in JavaScript then you better set aside a lot of time before writing production code in Haskell. These are issues that are more related to people than they are to programming.
The best language is the one that you/your team knows. The second best is the one that suits the task.
The only two to make me go "wow" were Lisp and Haskell. They really are that strange and different.
(Well, technically the assembly language we used also made me go "wow," in the sense that I had sympathy for the suffering of programmers in the olden days, but as an EE you may have a different takeaway.)
Thanks for point to Red, I'll have to check it out.
Yeah, I agree that it's about time we had more formal methods in software. OTOH, messing around with Dafny has been fun (well, I enjoyed abstract algebra in undergrad), but it takes forever to write anything useful.
[0]https://homepage.cs.uiowa.edu/~jgmorrs/eecs762f19/papers/fel...
Who didn't see this coming?
For someone who's constantly extolling the importance of other languages and flexible thinking, Paul doesn't seem to ever be able to get his mind out of Lisp.
> [Lisp macros] by their nature would be hard to implement properly in a language without turning it into a dialect of Lisp.
There are numerous counter examples to this claim, the most obvious one being Rust.
How do you call someone who only knows one language and refuses to learn new ones? Oh right, a blub programmer.
Lisp macros you define just like any Lisp function, and allow you to create new syntax constructs indistinguishable from any built-in constructs, which can execute whatever code they like as part of the process.
In Lisp, macros are evaluated at run-time and can do anything, including access program state (if you play your cards right). They consume something that’s very roughly an AST, and emit something that’s very roughly an AST, and the language makes doing that quite easy, largely because it’s all structural in nature.
In Rust, you’ve got macro_rules! which provides convenient syntax for simple structural macros, and then after that you have procedural macros which have the full power, but have to be written in a completely different place and get evaluated at compile-time only (so no dynamic macros), in a completely different context to the code that calls them (so no program state). They’re a lot harder to work with, consuming a stream of token trees (e.g. `if`, `3`, `.`, `{ … }`, `(…)`) and emitting a stream of token trees, a representation that’s clumsy and prone to error (especially since it’s not structural) though this can be mitigated a lot with crates like syn and quote which can make the process at least somewhat more structural, but it’s still way more clumsy than Lisp, where you can’t really distinguish between structural and procedural because they’re basically the same thing. Also Rust macros look different from built-in language constructs, all being like `foo!(…)`, `foo![…]` or `foo! { … }`.
I can’t comment about any other languages; they may have things much closer to Lisp macros.
(I write this as a Rust expert who writes structural macros regularly but procedural macros only once every year or two at least partly because of how painful they are to write, and a Lisp outsider who’s familiar with all the concepts and written a little here and there over the years but never anything serious.)
And just like LOOP in Common Lisp, you can totally make Rust macros that are a new language inside a language, and are too clever for their own good ;)
I also think it's a bit arrogant to imply that if you don't like Lisp than you're just not smart. I'm sure many of us have been exposed to Lisp and still prefer to programming language that less powerful but more humane.
I don't mean to be too dismissive. I think this short piece was thought-provoking enough to be worth reading, and it was not a bad argument in favor of Lisp. Maybe I really should learn Lisp. But if the heuristic was generalized from just one example, it might overfit to just one particular kind of weirdness.
My fear with lisp is that if you use macros it can become obscure what is really going on. Even in C# when I’ve seen metaprogramming done it can kill the discoverability of an app, meaning you have to talk to people who have talked to the person who left the company who knows the mental model. Searching code for keywords no longer helps, or you read the entire codebase to figure out how it works. It was a relief to get off that and work with more familiar CRUD like code. If someone writes a macro, they should write a manual!
You should try it out. The book Common Lisp Recipes by Edi Weitz is really good and practical - you can get web servers up and running in minutes, and has many other practical uses (I have to check the contents again, but its about 750 pages long and covers stuff like interfacing with C and Java), highly recommended if you want to get productive in Lisp!
The availability of programmers who are proficient in a programming language is the single most important factor in programming language adoption. I think the continued popularity of C++ and Java has much to do with the fact; an enterprise wants to treat programmers as interchangeable screws, and a less widespread programming language would make this practice much more difficult. C++ was designed to be easier to learn by means of backward compatibility with C; This decision was good for programming language adoption; it was a good trade off, even at the expense of being a source of many issues with the language.
Interesting implication: I think that an enterprise with a less common programming language (like Scala or Rust), would have to treat its programmers much better than a competing shop that is using a commonly used platform like Java or C++; they have a greater investment in their workforce, due to the language/platform issue, are probably more likely to raise salaries every now and then and would be less likely to 'hire and fire'...
So adopting a less common programming language might have some non trivial implications on the culture and governance of an enterprise.
On the other hand there would be a lot of pressure to switch back to a more common programming language, in the event that the shop needs to grow significantly, these shops would then be inclined to port their systems back to a more common language...
Using weird languages DOES have one merit though: it opens your horizons. It teaches you to see a "conventional" programming language as a collection of mechanisms at your disposal instead of something "intended to be used to do X".
More to the point, I think that there are two types of languages: 1) those that encourage top-down design (that's how I learned programming with Pascal) and 2) those that encourage bottom-up design (such as RPL, these have often an interactive REPL.)
So actively using something like RPL or ML or Scheme (yeah, I've read ca 2/3 rds of SICP) allows you to turn C# "inside out", to apply bottom-up programming techniques. You don't create a class because "some OO principles tell you to do so" or because it came as godsend from some "architect", but because you understand workings of classes and a class is the right tool for the task at hand.
Also, there's no programming in the large without some mechanisms to control accessibility.
I like macros. I find myself, not infrequently, thinking about where they would be convenient despite using a language without them.
Nevertheless none of the languages that include them (that I’m aware of) seem very appealing.
Part of this is “program-manipulating techniques”. Less powerful languages are easier for programs to reason statically about, and so usually have great editor refactoring / navigation / static analysis already available. This makes the 99.5% of programming that is mundane more pleasant, and it’s hard to imagine special programs that don’t involve at least a good chunk of this.
The alternative to macros is not never doing metaprogramming, you can do code generation, parse and process the AST, or have program behavior be data-directed at runtime up to including an embedded interpreter. These things are surely less convenient than macros, but it’s a trade off against making all the other parts of the program less convenient to write.
Another issue is most languages with macros are Lisp/Scheme types, and having linked-lists-of-boxed-objects be the default representation for everything often isn’t great for domains that are performance-sensitive (this probably wouldn’t apply to some languages like Nim).
Of course there are the other downsides of using uncommon languages, like often less mature runtimes / library ecosystems.
It seems technically feasible for a language with macros to address all these issues (and maybe Rust does if you’re willing to give up GC). If I was writing a new language I would probably start by compiling to either the JVM/.NET/Go to get a runtime and libraries, and focus on the problem of incremental language extension while maintaining good editor and static analysis support (I think there are some promising approaches for that but this comment is already pretty long).
Rust, Julia and Zig (if you consider comptime to be a specific type of macros) don't look Lispy to me but all have macros. In Rust I would consider them essential to the language (don't know the others well enough to say), without them Rust would be terribly verbose.
> having linked-lists-of-boxed-objects be the default representation for everything often isn’t great for domains that are performance-sensitive
Disagree, there are extremely performant Lisps out there, including Common Lisp and Chez Scheme. They run as fast as, or faster than, any language that's not a systems-language (C, C++, Rust, Zig).
Yes, that’s because CommonLisp also provides native high performance data structures like vectors, arrays and hash tables that complement basic linked lists. Furthermore, most Lisp compilers are pretty mature and create well-optimized x86 code. You can annotate your code with type hints to help and guide the compiler in hot spots of your program.
Sure, I even mentioned both Nim and Rust (Zig’s comptime is interesting but I wouldn’t put it on the same level). It just seems like most language implementations with macros are Lisp variants (although which ones count as different languages is somewhat fuzzy).
> there are extremely performant Lisps out there, including Common Lisp and Chez Scheme
If you look at for example the benchmarks game [1], Lisp puts in an extremely impressive showing relative to Python/Ruby but often lags slightly Java/C#/Go. But more importantly, the Lisp code is often very non-idiomatic [2], with lots of special annotations and even turning safety features off. Maybe it’s acceptable for optimizing very narrow hot spots, but I wouldn’t want to write a lot of code in this style and wouldn’t expect most third party libraries to be written that way.
I don’t think any of this is the fault of Lisp implementers, but more that the basically default (although not only) data type provides the semantics of dynamically typed linked lists, which are inherently (relatively) slow and tricky to optimize. And even if very clever optimizations exist, relying on them to get adequate performance is less than ideal, since it’s often not obvious where you fall off the fast path. Of course many applications aren’t performance sensitive enough for these sorts of things to matter much, I’m thinking about 3D games etc here where you usually want control of memory layout.
[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
[2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
The video is quite long (and has a Q&A accompanying followup), but I think just the example of creating custom iterators with macros is really interesting.
In general I really like the approach of designing a language in conjunction with implementing a large project (other than the language itself).
It sounds a little ungainly but parsing shouldn’t be a big percent of compile times, so the overhead shouldn’t be that big. And it addresses two of the biggest issues, libraries and (current and future) platform support, things that new language adopters should legitimately be concerned about.
A lot of the most popular mobile games actually now use a similar technology. The game engine Unity converts the bytecode for game C# scripts to C++ which it can feed into the system toolchain. Even as a billion dollar company they decided it was more practical to piggyback on another language for platform support (in their case game consoles, which come with their own C++ compiler but may not be supported out of the box by other language toolchains).
This seems to be true in the same way that "99.5% of cooking consists of making food hot. All popular kitchen appliances are equally good at this." is "true".
Technically you can fry bacon in a pressure cooker, but you shouldn't because we have frying pans.
I'm not sure this is so true. I've glued stuff together in a bunch of languages, and some are easier than others. Particularly in how the package manager works. I'd say c++ throws the weirdest integration issues out of the ones I've used. Rust seems pleasant, nothing too odd thus far for me. Python, super easy so long as you got Anaconda for all the math stuff. JS/npm maybe too easy. Swift/Cocoapods smooth sailing.
But also the languages themselves can make it easier or harder to glue stuff together. If you need to play with pointers and allocators, there's more work to do.
> Weird languages aren't weird by accident. Not the good ones, at least.
No true Scotsman? I'm sure once a language gets more than the initial attention, it can end up a strange mess, just like anything that has a bunch of people tugging at it.
> So if you want to expand your concept of what programming can be, one way to do it is by learning weird languages. Pick a language that most programmers consider weird but whose median user is smart, and then focus on the differences between this language and the intersection of popular languages.
I think this is selection at work. Yes, I do consider some coders better than others. But often it's because the kind of person who is good at coding is often the kind who gets to the bottom of things, and part of that is cave diving in weird languages. Probably I rate people higher as coders just from telling me they know Haskell or Lisp, which I perhaps shouldn't (?). But it does appear to me like some of the sharper coders I've met have for some reason been down the Haskell cave, perhaps investigating something to do with the type system.
Any suggestions on which languages could fit this criteria?
https://github.com/ziglang/zig/issues/130#issuecomment-70601...
> However, at this point the plan is to not add an additional dynamic dispatch language feature.
Each solution uses only a few language features. Different problems call for different subsets. Not using a feature on your favorite sort of problem says nothing about its value. "Ways to enable composition" poorly supported are reached for less. Poorly supporting one of them that has proven frequently useful does not improve a language.
There have been rather a lot of languages. The field is approaching 70 years old, and still changing, but clear evolutionary processes are seen repeated in hundreds of languages living and dead. The ones that don't grow are the most reliably dead.
Practical languages accumulate features to better address a multitude of common real-world problems. Toy languages don't. Sometimes a toy language is adopted and grown to practicality. People always complain about that.
I'm looking into APL and Lisp type languages basically to expand my mind as up to now I've really only being using "traditional" type languages.
It's like starting from scratch again.
I find "thinking in lisp, then writing the program out in X" where X is whatever 'mundane' language seems most suitable to the task is often a very effective technique.
Array languages I've tried to learn repeatedly over the years and always ended up bouncing off, so have no opinion there except "personal aha moment not yet reached".
Do they offer any meaningful differences to an otherwise "mainstream" language - anything beyond syntax, tooling ecosystem and the usual functional vs. procedural vs. OOP paradigms?
Anything I can't just whip up in modern C#, C++, Java or Python if I wanted to constrain myself to a specific subset of features or a specific paradigm?
I get the "it's fun to try something new every once in a while" part but people tend to forget that the distinctions between programming languages have blurred significantly over the last 10+ years.
- support for concurrency. After tasting it you never will want to go back to techniques normally used in the languages you listed
- data immutability, which makes reading code so much easier.
def test_square(side_length, area):
return (side_length ** 2) == area
:-use_module(library(clpfd)).
square_side_area(X, A) :-
X #> 0,
X*X #= A.
?- square_side_area(5,25).
true.
?- square_side_area(5,AREA).
AREA = 25.
?- square_side_area(SIDE,25).
SIDE = 5.
?- square_side_area(SIDE, AREA).
SIDE in 1..sup,
SIDE^2#=AREA,
AREA in 1..sup.
?- square_side_area(SIDE, AREA), AREA in 1..25, label([SIDE,AREA]).
SIDE = AREA, AREA = 1 ;
SIDE = 2,
AREA = 4 ;
SIDE = 3,
AREA = 9 ;
SIDE = 4,
AREA = 16 ;
SIDE = 5,
AREA = 25.
It isn't /only/ a built in brute-force search, or a "do everything with recursion" computer science lesson timewaster.
[1] https://pragprog.com/titles/7lang/seven-more-languages-in-se...
No keywords, only 2 statements (if/while), no strings in code but bulitin i18n. Here is EBNF grammar https://github.com/d08ble/acpu/blob/master/acpul-EBNF-cocor/...
Here is demo project https://github.com/web3cryptowallet/Web3CryptoWallet
But powerful like LISP and I am building mobile OS, IDE and nocode environment with ACPUL https://twitter.com/acpustudio
You'll also see which high level languages fit (or don't fit) this underlying fundamental model which may help cut through the marketing fluff from their various proponents and provide an unpleasant appreciation of the performance costs of popular programming paradigms.
What's weird: it allows a mix of declarative and imperative syntax in a style that seems like normal procedural programming. You have a "magical equals" that is actually reactive programming... if any of the terms change, ever, the result gets updated.
Here's a previous thread about it
I'll pick a seemingly random point:
> Pick a language that most programmers consider weird but whose median user is smart, and then focus on the differences between this language and the intersection of popular languages.
This heuristic used to apply to Python, but it no longer does. (Partly thanks to the the "Python Paradox" essay I suppose; most people now think learning Python will give you a nice career path).
All this heuristic says is that this language has some novelty that attracts people who are interested in novelty. People who are interested in novelty in the field of computers tend to be smart people, but they are not reprsentative of the "population of smart people working with computers"; they are just a small slice of it.
I might even go so far as to say, it's a slice of people who are just interested in things but not particularly interested in making finished products.
Most of the people who work with or on various linux distributions also fall in this category: they are smart, but they are not interested in building a solid working desktop environment as much as they are interested in just playing around with things for fun.
For them, their subject of interest is like a puzzle game that they spend their time exploring. There's no specific end goal. No specific product to be built. Nothin to be delivered to end users.
That's also the appeal of weird languages: the novelty itself.
This might be a matter of definitions, but to me a good language is one that allows you to deliver a solid reliable end product.
Maybe something you write in 5 lines of lisp will take 30 lines in Go. But this is not an issue at all.
The mental energy it takes to write the 30 lines of Go is probably lower than the mental energy is takes to write those 5 lines of lisp.
I would use more mental energy to write 30 lines of Go, than 5 in lisp. Also if you take into account that you have a true REPL and you can do REPL driven development, which spares you some extra mental energy.
But to be fair, Paul is talking about macros, which Go doesn't even have.Macros are more complicated, but are often oversold and over hyped. Yes, they are great, you can extend the language and all, but macros are not that often used, you can probably pick a random lisp codebase and search for macros, I bet you won't find any or less than 2% of the entire codebase. If macros in lisp didn't exist I would still use it, because I love building expressions vs building statemets.
One can look from another angle into some code bases:
https://github.com/McCLIM/McCLIM/blob/master/Examples/stopwa...
It uses a ton of complex macros:
Built in macros: DEFPACKAGE, DEFMETHOD, MULTIPLE-VALUE-BIND, LOOP, ...
Language extensions: WHEN-LET, ...
Library macros: WITH-OUTPUT-AS-PRESENTATION, FORMATTING-TABLE, DEFINE-APPLICATION-FRAME, ...
Even a generated macro: DEFINE-STOPWATCH-COMMAND (which was defined by DEFINE-APPLICATION-FRAME)
You change your mindset in different ways in each of them. When you come back to your usual tool you expanded your thinking to what you could not conceive previously.
Paul Graham also talks about the power of different programming languages in Beating the Averages, see The Blub Paradox: http://www.paulgraham.com/avg.html
Powerful / weird languages usually aren't the productivity hack you'd hope. A few team members might be more productive with a weird language, but other folks will have trouble understanding their code, so they'll be less productive.
Weird programming languages also require different types of library support (programmers that are passionate about immutability can't use a JSON library that mutates stuff). So they'll build a separate JSON library, but that fractures communities.
It's great to learn about different languages, but programmers seem susceptible to the allure of extra productivity from weird languages and that's not what I've seen.
This assumes, you know, that other folks are incapable of learning anything new.
Most engineers (as opposed to coders) are fairly results-oriented. We may have our differences about how to get the results, or even, what the desired results are; but we always have a deliverable in sight.
It's all about getting satisfactorily-functioning software into the hands of end-users. Put a bow on it. Stick a fork in it. It's done.
Speaking for myself, I write native Apple code (iOS/iPadOS, MacOS, WatchOS, and TVOS). I need to use Swift. I could use ObjC, but Swift is really where it's at, unless I'm doing really low-level stuff. Others may use JavaScript, or even C#, to write Apple software; using hybrid systems.
It is, indeed, mostly "gluing together calls to library functions," but there's a bit of algorithm, mixed in there, as well (not much. Many library functions provide better algorithm support than hand-rolled).
Other platforms have their most effective languages.
I probably could write Apple apps in FORTRAN. I think that there are FORTRAN compilers for Apple systems. I can use existing C or C++ code, to provide "business logic" (I used to run a shop that did just that), so I guess there's a lot of languages that I could use in the backend, and, maybe, some might be better choices, for certain tasks. Again, it needs to be results-driven. If a certain kind of algorithm is most effectively implemented in Lisp, wrapped in system calls, I could embed that in a Swift wrapper.
There's also something to be said for becoming a "native speaker" of a language. That means using it fairly exclusively, over and over again. Step and repeat. I have been writing Swift, almost daily, since the day it was announced, and I still discover new things, almost every day. I think that's pretty cool.
This is stuff that I would never have discovered on a "context switch." I've usually been fairly good at picking up new languages, but I am happy to be sticking with this one, for a while. In the past, I've had to know Pascal, Object Pascal, 68K Assembly, C, C++ (and simplified variants, thereof), and Objective-C, in order to program for Apple (yeah, I've been at it a while).
How about yourself? Do you still do programming in AI?
Camlp4, Template Haskell, and Rust procedural macros all serve as counterexamples to this claim.
Done properly? I can't speak to camlp4, but at least in the case of Haskell and Rust, certainly. Incidentally I just had my first occasion to write a Rust procedural macro last weekend. I had a significantly complex transformation written and working in half a day, learning curve included, and I found it all pretty frictionless.
If you have to parse anything, it's not a Lisp-like macro, which is always an object arising from a production in the language.
Parsing can be hard, so he is right there too. If we take out some subset of Rust that has a significant syntax, and implement it in macros, that sounds like being up to the elbows in parsing.
let ast = parse_macro_input!(input as Foo);
Just as use of `eval` tends to be discouraged in Lisp land, it's not something that a Haskell or Rust programmer would often reach for. But in Haskell, if you really want it, GHC has an API and you can have the whole power of the compiler available to you at runtime. This isn't really a language feature per se, it's literally just linking in the whole compiler and calling it like an ordinary library. I'm not aware of anything similar to that in Rust but I haven't really looked. However, if you're only trying to generate code and not JIT-compile or execute it, the same Rust crates that support compile-time AST manipulation (like `syn` and `quote`) can equally well be used at runtime.
Is this fundamentally different from "writing your own library to solve a problem, and then using it"?
> So if you want to expand your concept of what programming can be, one way to do it is by learning weird languages.
Isn't the conclusion here "contribute to/write your own libraries" rather than "pick a weird language"? Or "solve problems from scratch instead of relying on libraries when learning"? I feel like the "weird languages" are shoehorned into a discussion about glue programming vs library programming.
If you write your own library for a popular language, you're both competing with everyone else who has written a library to do the same thing and picking fights with those who just want tell you to not reinvent the wheel.
It is either that the problem is really hard and so most popular languages haven't attempted a solution and just make do with what they have.
Or, the problem just isn't that interesting. Like, the solution is cool, but applies very rarely and even then doesn't matter all that much. Popular languages could easily implement it, but choose not to.
At first, I was blown away how short the Haskell one was, but later someone wrote that actually, world-count-wise they were largely the same, and indeed he was right.
So in certain cases, ML languages are not terser, just more wide, rather than long.
Pulling numbers out of thin air like this to support an argument is counterproductive. This should read “Most of programming”, unless there’s research that can corroborate the number.
On the other hand, most of the world's most complex systems are built in boring choices like C, C++, Java, or Python. The claim that this or that can only have been built or get to market fast enough if it's in Lisp doesn't pass the smell test for me.
And having relevant libraries is (at least part of) "the merits".
Disclaimer: I'm the author
The issue is: Management of Complexity, not how to efficiently express complexity. This is why we have layers - layers are a kind of DSL; and this is where most complexity lies. Clear layering -- including defining the functions of each layer -- is how Complexity is tamed. (And this can be successfully done in assembly language, if you want.)
The main reason to master weird languages is: Fun.
This just comes across as arrogant. You can neither know how smart the median user is, nor does it say much about the merits of the tool. It encourages some weird sense of superiority by putting yourself in a group that's exclusive and think they're smarter than everyone else. There are better ways to pick programming languages than trying to maximise your own pretentiousness.
Paul Graham essays decline in value exponentially with the number of times he mentions Lisp in them. He had one that went below 2^-20.
Doing something outside of your bubble is great for learning some humility and G-d knows PG could do with some of that.
Even in lieu of that bad taste or elitism, I'd still like to suggest that it takes more patience, self discipline and perhaps a tinge of being clever to pick up something like C, C++, Rust, Haskell, Lisp, APL, Prolog or even regular expressions than it does Java, .NET, Node/JavaScript/TypeScript, Python and others and I say that as a person who relies on the latter for the most part in their daily life.
It's not that you have to be an outright genius to use any of them, but writing code in them well (for example, managing memory and avoiding C++ footguns) takes discipline and definitely could be applauded on some level, as could using APL, Prolog or regular expressions to efficiently solve problems in the few domains that they're actually good at and suited for.
Personally, some of these views of mine formed after listening to a podcast that talked about APL and someone's experience with it: https://corecursive.com/065-competitive-coding-with-conor-ho...
Just because the average developer in a language is bad it doesn't mean that the language itself is bad.
I'd say it depends on what we're defining "first programming languages" as - first PL learnt in intro-level university lecture, or first PL learnt individually? But my impression is that the most popular PL for beginners seems to be Python. And while I'm not really a fan of Python, I understand where that's coming from: Python is relatively simple to learn, has a huge library ecosystem and comes with relatively few footguns.
I feel like this is obviously true when taken at face value, however in practice there are probably loose correlations (not causation) between those two, even if not for the reasons that you'd expect.
I'm not sure whether it was mentioned in the same podcast that i linked, or another one, but there was a very interesting example for Haskell that was offered in one discussion. The argument was that the people who utilize Haskell tend to be more likely than the average developer to have ties to academia (which may incidentally also speak of their cognitive abilities):
- Haskell is popular for numerical processing, moreso than for the somewhat typical CRUD apps
- because of this, Haskell is more likely to have materials describing it which are aimed towards an academian audience
- ergo, those materials won't always be simple step by step tutorials on how to do things, but will sometimes assume familiarity with certain maths concepts and will also focus on solving a particular set of (advanced) problems
- and when these materials attempt to explain these simpler things, invariably you'll still end up with explanations of what monads are somewhere in there, whereas with something like Python you'd have less overhead and a less steep learning curve
- as a consequence of all of this, Haskell won't be as popular as Python, or similar languages
Now, the person might be wrong, since their observations are bound to be subjective and i might be wrong to assume that they hold true in many cases because of how believable all of that sounds, but to answer your question, i'd argue that the average user of Haskell is indeed a bit more intelligent than the average user of Java or many other general purpose languages (even though then we step on the mine of having to define what intelligence even is, where opinions can once again differ).
I'd say that simply happens, because some of those languages are far more approachable and therefore more widely used by a large audience of people, not all of which need to concern themselves with solving the issues that people in academia need to deal with. Of course, there could be the counterpoint that not everyone in academia is brilliant, but i remember seeing some pretty admirable professors and researchers back when i was in my university!
Some languages are difficult to learn initially (like Prolog) but that's not because they are intentionally obscure, but because they need to be different.
- programming language problems like making certain language features, paradigms and kinds of abstractions available (or not available)
or
- solving actual problems when building software - how to build a maintainable and performant server, OS, web UI etc.
For the second you're better off staying away from the 'weird' languages especially if you care about being productive.
The way I think about programming changed a lot from having used different programming paradigms. Many programming paradigms have little real-world value, but they do make you think about computation, abstraction, and lots of other concepts differently.
That grows your brain in ways which are useful when you go back to your 9-5 Python/JavaScript/Java/C#/etc.
I no longer have time to do these sorts of deep dives, but I did this a lot when I was young, and I'm a much better programmer for it.
I'd look at what he's writing over how he communicates it. Perhaps the language might be arrogant or self-congratulatory, but it benefits you more to look at what he's saying than how he says it. (responding to thread here; you obviously didn't make this mistake)
However I am not going to argue that the median javascript user is as smart as the median ocaml user. 99% of people learn javascript because they have to. Most people learn ocaml because they are enthusiasts. They're interested in programming language concepts and becoming more effective.
I would never call it a "good" language. It is an abomination created by the dev teams at 4 different browser companies over the course of 20 years with no coordination between them.
It is the swiss army knife of programming. Even top-tier hackers use it as a primary tool. Check out Frida[1], it uses the Chrome v8 engine to inject parasitic tracing code written in JavaScript into any OS (incl. android & ios). There's no language that would have fit the bill like JS+v8.
- Relatively fast
- Enormous ecosystem - the "good parts" of which still have practically everything you need
- Top notch typechecker in the form of typescript
I interpreted it at second effort as:
"Pick a language that is famous for being an oddball but that has good credentials, since you have seen that people that to you are worth credit use it".
I’m reminded of his argument about how “hackers” choose languages that let them type the fewest characters possible, an argument that seems empirically unfounded to me.
Ah yes, just like real writers write in Ithkuil, a language constructed to maximize the meaning encoded per syllable.
It’s fine if that’s his preference, but the argument that the best programmers (“hackers” in his parlance) prefer what he prefers to be both self serving and contrary to actual real world evidence.
I sort of get the sense that he doesn't realize how many crazy talented (but not famous) programmers there are out there. He's got a microphone and is determined to keep using it to tell us how great his ideas are, but the audience has gotten a lot more discerning since the 90s when producing a CRUD app in Lisp could still be considered a feat.
Once upon a time (about 1992ish) in a land where the moles
have beaks and swim and the deer jump on their hind legs
(Australia), a certain computer science department (RMIT)
decided that Pascal had reached its use-by date. What shall
we do? What shall we replace it with? Set up a committee!
I was on the committee. We set up a short list.
(1) Scheme.
Tiny language, amazingly capable, REPL so you can try
things out, implementations for all the machines we
cared about. And Rob Hagan at Monash had shown that
you could teach students more COBOL with one semester
of Scheme and one semester of COBOL than you could
with three semesters of COBOL.
(2) Miranda.
The commercial precursor of Haskell.
Melbourne University were also looking at this (or had
already switched to it) which would make it easier to
pick up some of their students.
(3) Ada.
Close enough to Pascal that our staff were comfortable
with it and our material would not need major revision.
A better Pascal than Pascal.
Handled concurrency about as nicely as an
imperative language can.
What was the deciding factor?
Schools.
Our deliberations leaked, and we started getting phone calls from
careers masters saying "if you go all theoretical [read: (1) or (2)]
we'll tell our pupils not to study with you."
Personally, I'd maybe not apply the adjective "smartness" here, but there is some sense in which some languages have better taste than others, and what drives language popularity is not usually good taste, and I interpret pg here to be saying he prefers to recruit programmers who show good taste in the languages they work with.
I don't have any sort of legitimate control group to compare to, and I'm obviously biased as heck, but I do still hold the impression that I and others who learned that way can learn new languages and programming techniques much more quickly than acquaintances who were fed a steady diet of Java or Python. Many of them, even 20 years on, still don't really have a firm grasp of what single dispatch is in the first place, which hinders their ability to use it well. Which is ironic, considering it's all they've ever known.
I agree; I hate the "smart" perception. My guess is that, if there's anything there, it's just the gift of a stronger grasp of the fundamentals. Scheme brings you very close to the abstract foundation of computer science. C brings you very close to the technical foundation of computer engineering. I'd like to think there's a lot of value in starting out by establishing a firm foundation like that.
Dr. Richard A. O'Keefe is a computer scientist best known for writing the influential book on Prolog programming, The Craft of Prolog.[1] He was a lecturer and researcher at the Department of Computer Science at the University of Otago in Dunedin, New Zealand and concentrates on languages for logic programming and functional programming (including Prolog, Haskell, and Erlang).
As a student, we were required to learn assembly in a microcontrollers course. It was a disaster, because it was a very short semester and they were asking we learn microcontrollers and assembly at the same time when most of us could barely program to begin with. I left a review urging they change to C (still low level, but so much easier) and they actually switched the next year.
Learning ASM is indeed important from a theoretical stance (and practical for some jobs), but you can't expect students to just pick that up in a few weeks worth of classes.
Then when I got an office job, I learned a LOT of engineers do process automation programming. At that point I could see even something as hideous as VBA being used to efficiently save hundreds of hours of work.
What do students really need? It depends on a lot of factors. Are they going into academia to study PL theory? Then Lisp, Prolog, C, and Haskell are probably pretty important. Are 95% of your students going into industry? Then Python, C++, Java, C#, JS, Excel, and SQL are all going to be important.
I think the really theoretical stuff (Haskell, Prolog, Lisp) should only be briefly introduced in undergrad and expanded upon in grad school. Or you have specific academic or industry based programs.
My experience is Haskell actually is highly practical for many tasks. So I think it’s misleading to call it “really theoretical”. It’s the reason you see FP in areas as diverse as spam filtering, distributed computing, smart contracts, various financial services, etc.
While less common, there absolutely are areas in industry that do tend to use FP, and I don’t think you should cut students off from easily entering those specialties (or having to plan to enter them ahead of time).
Additionally, there are concepts used in imperative languages which are not as easy to understand or express in those languages, e.g. promises, are more clearly expressed as monads in a language like Haskell.
It’s really funny you say that, I view HN as being frequently anti-FP. I guess when you split the difference everyone is a bit unhappy. In my case I’ve been programming for over 30 years and only on HN for a few months, so I don’t feel like it’s swayed me much, you never know. I’ve read Graham for years and he certainly has been an influence.
Overall Haskell alone isn’t that big in industry, but it most clearly illustrates many FP concepts and is the originating point (or an early adopter) of new tech that makes it into other languages.
If you look at all the jobs in Scala, Haskell, Clojure, F#, OCaml, Elm, Erlang, and PureScript. Then you throw in all the jobs in multi-paradigm languages Python, JS, etc. with managers or coding standards that dictate functional techniques, then subtract the particularly awful soul crushing dead-end jobs (that regardless of paradigm or language) probably none of us want, I think you may be underestimating the representation of FP.
Ultimately when I hire someone with a 4 year degree from a research university, I’m not expecting them to have 100% vocational training, they should be literate in at least a couple languages, but I’m much more interested in knowledge of concepts and adaptability.
The best Python programmer I’ve worked with described his junior year as being almost entirely Standard ML. I think he was better for it.
> And Rob Hagan at Monash had shown that you could teach students more COBOL with one semester of Scheme and one semester of COBOL than you could with three semesters of COBOL.
Really makes the point about how studying uncommon languages can pay off.
Every single programmer interested in any language in the ML family or Haskell has considerably above average[1] intellectual horsepower. The syntax and type system can both fairly be called "weird."
The Lisp programmers I've encountered are along a wider range. They've ranged from average to ML tier, the latter being the ones that, as PG observes, are more interested in "weird" features like macros, call/cc, and so on.
And then you have extremely popular languages like C, C++, Java, JavaScript, Python, and so on. Virtually every programmer has used one or more of them to a significant degree, so the mean programmer using one of those languages is virtually the same as the mean programmer in the general population. This is expected since in cases like this the mean of a subset approaches the mean of its superset as the size of the latter approaches that of the former. Programmers who master C++'s template system are an exception to the above. They're also well above average.
[1] In this comment everywhere I say average or mean I'm referring to programmers and not the general population. Average isn't bad, the average person here is quite bright for example.
Just to point that there's nothing strange or unexpected about this. You will get people that are more intellectual than average it you filter by any niche intellectual activity with no immediate non-intellectual reward.
I mean, if you search for programmers that understand plant physiology, you will get the same effect. It basically means that smart people tend to be curious.
I'd say that since there's lack of Haskell jobs, then non passionate people do not touch it since it's useless to them
Passionate people check it out cuz other passionate people recommended it
And circle repeats.
Whik I'm not sure I'd always agree with PG, a recent anecdata I heard was a company selecting for Lisp because picking Python meant wading through mountains of fresh bootcamp grads with (quote) "overblown sense of their own capability" (unquote) - same applied to JS, less so to C#/Java.
(Un)fortunately, the company noticed they didn't have resources to filter that, and decided that by going for "weird but capable" combined with global remote organisation they could filter for people experienced, or willing to experiment with weirder languages. Apparently ithe cost of having to sometimes implement things from scratch was lower than expected.
Niche languages, by definition, are not good investment if you're after a well-paying job. A niche language that requires understanding advanced programming concepts is, by definition, filtering for people who a) can understand those concepts, and b) have a non-monetary reason for doing so, which implies c) they're likely not faking it.
Whether or not such people would be good software engineers is another topic. Software engineering encompasses more skills than just raw programming ability. But a proficiency in advanced niche language plus proficiency in a mainstream language plus some work experience? It's hard to imagine how this would not be a good signal for smarts and skills.
If I see just one smarter than me person using a particular tool for some task that I also do, it might be because they are using it as an intellectual challenge or for variety, but if I see a lot of smarter than me people using it for that task I've almost always found it is worth looking into it seriously to see if I should use it too.
“Pick a language that most programmers consider weird but whose median user is curious and shows the initiative to explore strange new worlds of how to get things done.”
I consider myself a very capable hack. I am not really very “smart.” But I am motivated to learn more. And to improve the craft of software solutions.
I believe I learned a lot, and most of the people I encountered on those slacks and forums seemed a lot more experienced and capable. That's also because the language is kind of niche but powerful, so not everyone is starting out their career trying to print Hello World in Erlang. It is a language one would discover when trying to solve a specific problem, in my opinion at least. Therefore, there are less noobs like me and the people I encountered were super helpful and thorough compared to my experience on other languages.
I also see no pretentiousness. With smart people hundreds of useless discussions are avoided in the first place. If you work in a popular language like Python that attracts all sorts of people, you have 10 preposterous technical and social discussions per day.
I think this essay isn't supposed to be anything comprehensive about Lisp, it's just encouraging his target audience to consider other options.
But, many years later, I’ve learned I gain nothing from this kind of defensiveness, and simply keep a running list of things I’m “supposed” to know, but don’t yet. Of course, now that is a small list of very uncommon things, which itself shows how much I’ve progressed.
Point being, digging in your heels only hurts yourself.
This is surely the key line here - and the stuff about glue code and median users etc frankly just detracts from it.
It's true in any field. If you want to expand your ways of thinking look for stuff that has been reasonably successful but is outside the mainstream. Put it that way it's sort of obvious but probably easy to forget.
Having read about this, the best analogy I can think of is that macros are like engineered wood. If you're building a structure, you can build bigger ones with engineered wood, than with run of the mill (literally) stuff.
For example, let's say your language had 'if' but not 'case', then it would be possible to write a case macro that in the end transforms at compile time into a bunch of ifs. Macros allows you to create new language syntax and express your intent more clearly.
There are many other nifty things you can do as a result, but I am myself not that great. Paul Graham actually wrote the book on macros ('On Lisp'), he's legit :-)
I'd love to see an expanded version of this article written by someone less focused on Lisp advocacy than Paul Graham seems to be.
Incredible projects like Guix, Emacs and Datomic are proof that Lisp is still alive and well and can create great things.
I would give Prolog or maybe Forth as examples of weird language that still have a "smart userbase".
For what its worth, he's writing his own Lisps (Bel, Arc), is highly proficient in Scheme (just check out the book On Lisp), and is a true lisp polygot. I can't speak for him obviously but I don't think he has a super strong affiliation _just_ to Common Lisp.
In terms of influence and earning potential, parochial and closed languages like Python are kind of a pyramid scheme:
The people who were there first occupy many of the top positions in industry, regardless of their contributions or competence. They actively harm and intrigue against dissenters, so be prepared to chant the religion of the day (which the powerful change from time to time).
This sort of thing cannot happen in languages with a standard and multiple implementations like Lisp or C++. Haskell and OCaml seem to be isolated from the most petty forms of intrigue because the average contributors are way smarter and care more about actual technology.
The only thing his less succinct essays give you is his motivation for liking Lisp and macros in particular - it allowed him to build a successful business that gave him his big break in life.
or write compiler and experience the freedom
Stuff like "make this gql query, but with account spelled 'akkounte' b/c it's in language X, compute and add tax, then depending on the associated customer's SAAS, maybe stuff it into a REST call that has the account and tax as a CSV.
Or: convert this list of integers into f64, take the running average with a window of 5 entries (unless there's a holiday that week) then shove it into this C function that someone else wrote
This is also a statement about what kind of programming the speaker has done.
For me, using different languages is a statement that I enjoy programming in different languages, and not a value judgement. There are so many great languages and I think some new languages like Swift and Rust are especially fun and useful.
Programming languages will not become irrelevant but would be more akin to the fate of humanities, likely a personal passion. The economic value of learning programming languages in the distant future would be close to none.
Programmers should view languages as nothing more than a tool to perform some tasks.
Conversely, I guess ruby, node, python, perl? all have fairly standard package managers so it's generally? easy to grab a library and it just works, no having to learn how to integrate it into your build system, only having to learn how to use its API
I do agree there are big things to learn from different languages. I haven't actually used F# but I did learn from this article (https://fsharpforfunandprofit.com/posts/designing-for-correc...), that I'm missing out on saving myself a ton of work by not using a better language that (a) builds new types for me easily (b) lets me switch on types easily.
I'm also mixed on lisp's macros. What I'd like is a language that can use itself at compile time. I hate that nearly every large C++ project integrates other languages (make, cmake, gn, python) somewhere in the build process. With a language that can run itself at compile time like lisp the build tools themselves can also easily be in lisp. I don't thing C/C++ though is particular good at simple string manipulation so using it to build itself sounds like it wouldn't be so nice. That might be another example where some languages excel over others in certain domains.
The downside to lisp macros is everyone builds their own DSLs and so I have to learn those DSLs. Of course maybe that's just the same as learning some C++ class hierarchy I'm unfamiliar or some template magic so possibly not a valid complaint. It's just a feeling from the lisp I've worked with.
“A language that doesn't affect the way you think about programming, is not worth knowing.” - Alan Perlis
Pick and combine and switch as you go, tweak and create new ones if the availables aren't sufficient or are inaccessible. Worshipping and mantras are dead end.
It's that simple. (btw, a house is a tool to live inside it.. there was some Peter-Gabriel-wrote-music movie around this)
have fun
This whole article just feels like it is talking about code for its own sake, which I admit may be intellectually interesting, but has little to do with what we actually do day-to-day when building software.
Maybe that was really the point he was trying to get to - that building software is not the same thing as doing "interesting" code.
"there exist mental frameworks much more expansive for one's intellectual keyring than conditionals and loops («gluing [...] library functions») - for example «Lisp macros» -, so, «if you want to expand your concept of what programming can be» [presumably to acquire further hold of complexity, and new ways to navigate the solutions space], then explore weird languages with credentials".
In my view this programming language grand standing hasn’t been compelling for a long time, and now just seems kind of silly. Unless you’re working on really interesting problems, which the vast majority aren’t, judging from recent yc classes.
And there was data that showed that like 90%+ of YC startups use either Ruby (often Rails) and Python, including all the most valuable ones like AirBNB, Stripe, Doordash, etc.
So basically it's been 15 years since Ruby and Python have been acceptable (or even better) Lisps, according to PG's own metrics.
Ruby and Python both have garbage collection, REPLs, and enough reflection/metaprogramming to "compress" your code.
https://charliereese.ca/article/top-50-y-combinator-tech-sta...
For example it says Lisp for reddit but today it's Python.
Same goes for Smalltalk in this regard.
I like Python a lot but I miss some kind of functionality like that.
If not then... isn't it ironic?
Actually, different languages do make this part different. Because you might glue together something other than "call", i.e. good functional languages have abstractions that blur the code and data boundary.
If all interesting ideas were at the expense of code reuse, learning new programming languages would be a fools errand. Improving the last mile of whatever you are doing is an O(1) improvement of total cost — it doesn't even show up in productivity expressed as a rate.
The real compelling belief behind programming languages is much more exciting, namely that we can express better ideas together in reusable ways, invest that productivity into better languages and rolls, and repeat that process ad infinitum. This is a crazy railgun of a concept.
Incidentally, this is why startups as PG likes are a failed way to advance technology. Startups, focused on a limited 5 year window and that last mile, are structurally incapable of working on generalized infrastructure. The proliferation of b2b over b2c on recent years belies that, but replacing the current ossified FOSS Monopoly (our commons) with b2b bulkanization could restore competition in the means of production, but that bulkanization will also prevent the code reuse that create enough payoff to power the continued reinvestmemt.
Tl;Dr PG sees good languages as a way to do one-off great man history-style heroic inventions, I see good languages as a way to unlock factorio-style continuous high capital investment.
here we go again