Hacker News Reader

Comment by dang | Hacker News Reader

Making a programming language specifically for the needs of one program, then developing that language and the program together, is an underestimated strategy. I believe the world would be more interesting if more people applied it—because then we'd get more qualitatively different new systems. The Sapir-Whorf hypothesis may not be something people currently believe about natural language but for sure it's true about programming: the language you program in conditions what you think, which conditions what program you write. When the two evolve together, evolution can go to new places.

This strategy is time-honored in the Lisp world, where making a language and writing a program are more intertwined, and the cost of making a language much lower, than they usually are.

View on HN

The downside to this is that unless your language gets widespread adoption, you can corner your community into a bubble and drastically increase the barrier to contribution.

I think one of the issues of Gnome Project is this, they use a language specifically designed for Gnome/GTK+ and as if that was not enough, it has two different syntax, one--Vala which is sort-of-kind-of C#-like and the other being Genie which is sort of kind of python-like.

From the website:

> There's also an option to generate C code to support more platforms and achieve [(possibly)] better performance

Another reason x-compilation to C matters: providing an escape in case the community thins out and no further updates are made on the V lang.

That depends heavily on what the transpiled C looks like, machine-generated code can be pretty human-unreadable. In particular you generally lose all comments, you can end up with autogenerated variable and function names, huge functions with completely unidiomatic C code etc...

For instance here's what the chicken scheme transpiler outputs when told to compile a simple '(print "Hello, world!")': https://gist.github.com/simias/c96408a76eb7288fbd0d975f5dc99...

Good luck maintaining that...

I agree that modifying Chicken's output like you would handwritten C code would be an exercise in madness. But it's possible to design a language to have a nice transpilation story, for example PureScript deliberately puts restrictions on the names of types and operators so that they have a sane name in JS. Of course there's still a likely style mismatch, but what can you do?

I suspect you already know this, but for the benefit of others.

And looking at V, which is the story here, it should not be too obfuscated.

I tried to make resulting C code very readable and simple:

fn main() { println('hello world') } ==>

int main() { printf("hello world\n"); return 0 }

> unless your language gets widespread adoption, you can corner your community into a bubble and drastically increase the barrier to contribution.

Only if your language (and standard library) is of comparable complexity to that of widespread languages. C has more quirks than it lets on, C++ is a monstrously complex beast, Java has an enormous standard library…

If however you keep the syntax and semantics of your language simple, they can be learned in a matter of minutes. If you keep the "standard" library focused towards your application, it won't require more effort than any other regular application.

The real problem with custom languages, I think, is that very few programmers can actually write one. Most others don't even see the need, I think in part because of motivated cognition (If I delude myself into thinking I don't need something, I don't have to face the fact that I can't do it). Though the main problem is probably education: we are taught that languages are chosen, not made. As for how they are made… well, that's the realm of geniuses who have way too much time to spare.

Every programmer should have an introduction to programming languages, and we need more specialists who can whip up a DSL in a couple days. Our craft would be very different (and I think much better) if we did that.

Elixir allows you to build powerful DSLs within the language using macros and optional syntax. Also, if you write CSS well you end up with a DSL describing your screens. What other languages give you this power?

The Tool Command Language and Toolkit (Tcl/Tk).

All Lisps and schemes?

What other languages give you this power?

Rebol / Red

NB. Here's an old Wikipedia link listing (some) languages that were good for creating dialects (DSLs) - https://web.archive.org/web/20140708161345/http://en.wikiped...

Red seems deeply underappreciated in my view. Any thoughts on why?

Red is still at alpha stage of development. I suspect once it reaches beta level then it main garner more appreciation.

When it last showed up here in HN, iI was very intrigued by it. However, it promises a lot of stuff I think many here have heard other projects make. It seems very ambitious. Documentation isn't the best, which makes it harder (especially due to different syntax). Overall though, I am still a little skeptical it can keep it's promises. (With that said, I hope it does: it looks like it has a lot of potential)

It's still an alpha language. I would never deploy something in production with it.

Scala. For better or worse.

Having wrestled with the ~ and {} spaghetti of a Spray routing module... for worse

On the other hand the parser combinator library does it pretty well. Scala gives you enough rope to make a raft.

Terra [1]. I used it to create Regent [2] specifically because it takes this approach.

[1]: http://terralang.org/

[2]: http://regent-lang.org/

You’re affiliated with Stanford? If so, also with the developer of Terra?

To add to the chorus, most ML dialects, Forth, Ruby, Self, Smalltalk, Python, Mozart/Oz, D, Julia, Lua; to a significant extent Perl, PHP, Nemerle, Octave, R, Javascript, shell, even venerable old SNOBOL.

Hence why making a new application-specific, general purpose language is probably not the greatest idea vs building a DSL within one of the languages you mentionned.

Simplicity is undervalued quality. It's far simpler from the point of view of dependency management, that if one is developing a large codebase in C++, to also have the scripting language be native to that environment, rather than to first create a DLL of the C++ codebase, then implement the FFI to it using e.g. SWIG, PINVOKE, or such, to call the C++ context from one of these "premade DLL" substrates.

It's fairly trivial to expose C++ libraries to any other language context, IFF the library is wrapped to a DLL that respects the C ABI. The reverse is not true. You need to have some message passing mechanism in that case (yes, there are several not-so-hard ways to do that but that again, is added complexity).

> Simplicity is undervalued quality.

Yes, and building a library (a DSL) in an existing powerful language that can do it (and there are many as people said in this thread) for exactly what you need, is _vastly_ simpler than making a new language toolchain (which linker? which codegen? which typing?).

I agree - choose the best tool for the job. As a counterpoint - Implementing an interpeted macro-less lisp is a few thousands of lines of code in C or C++. So if the goal is just to bootstrap some language, the effort is more about typing than design or innovation (just copy the interpreter in chapter 4.1 of Structure and interpretation of computer programs).

A stack based language like forth would be even less work.

OOP is based, partly, around the idea of 'mini algebras' (roughly speaking, public interfaces), which are domain-specific languages ( http://xahlee.info/comp/Alan_Kay_on_object_oriented_programi... )

Functional programming, especially Haskell, tends to use the term 'embedded domain-specific language' ( http://wiki.c2.com/?EmbeddedDomainSpecificLanguage )

Lisps tend to call them DSLs or sometimes "mini languages"

Forth calls them "vocabularies"

I thought Lisps called them "dialects" (same as in Rebol / Red).

Couple of other terms i've seen...

* Pidgin - An attempt to get away from DSL ambiguity!

* Slang - This is what Perl6 calls them

> I thought Lisps called them "dialects"

(Lisp) "dialect" seems to refer to a particular implementation/variant of Lisp, e.g. CommonLisp, Scheme, Racket, Clojure, MacLisp, NewLisp, etc. rather than a domain-specific language built in a Lisp.

(e.g. searching for "lisp dialect" gives pages like https://www.slant.co/topics/5928/~lisp-dialects )

I've written several DSLs in Nim[1] for emulators & VM, JITs, parallel computing, neural networks and parallel tensor operations.

[1]: https://nim-lang.org/

D does that quite routinely too.

But powerful languages of today let you build DSLs within them, so there is hardly a need to write a lexer/parser/semantics for a whole new language.

That would certainly be a good way to do it (though I reckon the limit between a mere library and an internal DSL is a bit fuzzy). Do you however have any widespread powerful language in mind? I don't know of any. Even all of them combined probably still doesn't count as "widespread".

Also, once you understand the problem space well enough, a custom syntax can be a nice bonus.

I would say that Scala can be called widespread and allows for such embedded DSLs.

Genie seems super dead. Also, Vala is not that important for Gnome. Many apps are written in C, JavaScript, Python and Rust.

Erlang came about like that, starting from Prolog with a telecoms program and new language in mind, and developing both together. Prolog (I learnt recently) is surprisingly Lispy, e.g. data and program in the same form and easy for the program to change, everything in lists, and very FP - recursion for loops, non-mutating variables etc.

I don't how many other languages started like that. PHP for one. Probably a lot.

http://www.math.bas.bg/softeng/bantchev/place/erlang/a-histo...

http://www1.erlang.se/publications/icfp97.pdf

PHP is relatively infamous for being badly designed and hard to maintain (something I did professionally for years, so I'm speaking from experience).

Erlang is still a niche.

Both are examples of confirmation bias. There is a massive graveyard of one-man languages that no one used and then died.

That was true up to PHP5, where a Java-like, saner subset of the language with stronger typing became available.

The genius of PHP was in the code delivery model and the way you could intermix HTML and code in a single page to be served by Apache.

That democratized Web programming and programming at large. The rest of the language was bad, but it didn't matter for success (see "worse is better", where "better" means fitter for the market).

If you’re saying that PHP was a useful, Turing-complete templating language, I agree. I don’t think it was well-suited for large code bases with lots of business logic, and these days I’m horrified by the idea of a Turing-complete templating language in the first place.

Turing completeness doesn't bother me as much as all the security issues that existed by default (some of which are still present to this day).

I wish programming languages made a type-level distinction between strings that are and aren't tainted by user input. That would make it so much harder to accidentally introduce injection vectors.

I totally agree that PHP was not well suited for writing large and complex programs, but its ubiquitous availability as a deployment platform made it an interesting target anyway.

The easy deployment also made it easy for people to learn it by themselves, and consequently there was a large pool of programmers who knew PHP, making it an interesting language from a hiring point of view.

And that's how we ended up with piles of crappy, proprietary and OSS PHP code in the wild. The quality of the language is only a minor factor its success.

> The easy deployment...

This is an important point that is still underappreciated even today with all of our hindsight.

Confirmation of what?

I was just trying to think of (well-known) languages written that way. I guess there've been just as many dead languages not written with a particular program in mind.

What I took away from this, is “everyone should learn a Lisp dialect ASAP”.

Not necessarily. It's easier in Lisp, and a decades-long tradition, but Lisp is not even the only language of which that's true—there's also Forth. And there are many opportunities to apply this approach beyond those two. The more the better, because each qualitatively different starting point will lead to qualitatively different systems.

The software ecosystem is not well served by everyone being focused on the same few familiar programming approaches. Sure there are economies of scale—better tooling, programmer fungibility—but there is less intellectual diversity. More people should realize the tradeoffs here: yes you lose a lot when you start making your own language, but you also gain a lot. The program you're trying to write becomes more writeable. If that doesn't sound significant, it's because we're so conditioned to think the other way. There's another advantage, too: it's deeply intellectually rewarding. That provides staying power to work on significant projects over the long haul and is a solid motivation to do something many people think is crazy.

Really you can write a DSL in almost any language. I've worked with Java and TypeScript Selenium testing suites that were so project specific with everything abstracted out to methods so much you could barely tell what language you were writing, just that it was some C-descendent

You can, but having done both I came to the conclusion that they are far from the same.

> The software ecosystem is not well served by everyone being focused on the same few familiar programming approaches. Sure there are economies of scale—better tooling, programmer fungibility—but there is less intellectual diversity.

Then it seems that the problem has more to do with political economy than with programmer culture. Capital and managers want programmers to be cogs, not artisans.

Can’t expect programmers to change this political problem, either. Especially considering that the nerdier the technologist, the less political he or she is.

Or OMeta http://tinlizzie.org/ometa/ or Ohm https://github.com/harc/ohm or the like

Writing a simple TDD kata in OMeta was really mind opening.

That and Maude is another fascinating language. Pure term re-writing and explicit definitions of all data structures seems almost like a higher level of abstraction or programming.

[1] - https://en.m.wikipedia.org/wiki/Maude_system

I think for specialized projects it makes sense and I can think off the top of my head of two: Godot and Processing. Godot is a game engine that is highly extensible, and has 1 main scripting language reminiscent of Python and another that's for lower level patching of other languages into the engine.

Processing on the other hand is more of a way for visual people to get into programming and do visual representations of data or art. A matter of fact the demo for V / Volt made me think of Processing and finding out that it can be cross platform made me wish Processing would produce native binaries like this that compile down to very small executables whilst still maintaining the same syntax (which is very Java-like, but not entirely necessary to reproduce a full on Java-like language for this purpose).

I totally agree, for certain projects it makes a lot of sense to build your own language. I wish those mainstream engines would stop embedding C# (and I love C#) and make something much more creative. There's the not-so-mainstream engines out there too that do implement their own languages like GameMaker and friends. DarkBasic also comes to mind.

Yes, co-development of languages with applications of language is important.

Not sure about “one”, though, Usually you want more than one example before you generalize to a library -> framework -> language.

As one of my professors used to say (a lot):

“Man kann nur von etwas abstrahieren das man verstanden hat”

You can only abstract from something you have understood. And premature abstraction is probably one of our biggest problems.

What a terrible idea. I'm surprised I need to spell out that the benefit of learning a language is that you can actually use it, and the effort-reward ratio is far better if you can reuse it widely. Plus a language is far more than its syntax and compiler - its the ecosystem that makes a language productive. The last thing we need is for an explosion of half-implemented, poorly supported toy languages for individual programs.

I disagree. A cambrian explosion of 'toy' languages is exactly what we need so that the few that become popular reach the maturity you desire.

I have this itch that there is an explicit slot in our native language infrastructure for a scripting language that is a bit less clunky than Lua, not so huge as Python, and more ClojureScript like than the other embedded lispies out there, that is trivial to embed, has a fantastic interface to C++ and is utterly pragmatic yet elegant.

A small subset of Python would probably do...

Cython is great, and is a subset for native compile/interop that is often overlooked.

I agree on principle. However, in the comment above I was thinking about a scripting language that is trivial to bolt on to an existing native program, by including just a few header and source files. Like Lua, Duktape, etc. Or how sqlite is distributed in amalgamated form.

You might be interested in Starlark [sic], https://docs.bazel.build/versions/master/skylark/language.ht...

Unless you are the competitors.

I agree but at least some of the ecosystem concerns can be mitigated if you choose the right technologies to build your language. For example Eclipse Xtext lets you design languages that can easily interact with Java code and tooling, and it provides a lot tooling support like build integration in the IDE, syntax highlighting and code completions without much effort. If you keep the scope of the language small and treat it like any other module in your projects than its not hard to maintain.

Meta-circular composition forms a powerful design technique with more than just algorithms and code. The philosophy of affordances, fault prevention, fault recovery. Self similarity helps with sourcing, alignment, jigs, etc. Many design patterns in code have analogs in other forms of engineering.

Creating a language is in an orthogonal dimension to the plane that Sapir-Whorf lives on. The realization that you can create and utilize constructed languages, not just the given languages, to solve problems. A computer programming language is a meta-tool, one uses the language to shape and reify ideas. So a computer programmer is a meta-tool user. Someone who creates computer languages is a meta-tool creator. And they are designing an idea that is congruent mathematically, mechanically (can we compute it) and mentally for a human. Maybe my version of the Turing test is, "design me a computer language to represent and solve X"

> [...] developing that language and the program together, is an underestimated strategy. I believe the world would be more interesting if more people applied it [...]

I find this applies to most development, once you muddy the distinction between program and language you can see how the concept applies to other things - that is, it's a continuum, and artificially and prematurely locking down the design and implementation of a lower level piece before exploring the landscape can hurt... though determining what is "premature" and what is not is of course pure intuition.

> Making a programming language specifically for the needs of one program, then developing that language and the program together, is an underestimated strategy. I believe the world would be more interesting if more people applied it—because then we'd get more qualitatively different new systems.

One example of this is video game designer and programmer Jonathan Blow, best known for creating the game Braid.

https://en.wikipedia.org/wiki/Braid_(video_game)

http://braid-game.com/

He's working on a language for writing games.

https://www.gamesindustry.biz/articles/2018-07-02-jonathan-b...

https://www.reddit.com/r/programming/comments/7okvam/jai_lan...

https://www.reddit.com/r/programming/comments/8ywd77/gamelab...

https://www.reddit.com/r/programming/comments/89209l/jonatha...

Here is one of many videos of him using the language:

https://www.youtube.com/watch?v=AAFkdrP1CHQ

A couple of moments in that video which I would like to highlight:

At 11:15, responding to a question from chat:

> Is there a specific reason the game is kind of in top-down 3D?

> Because that is what helps the game mechanics that we need for this game. If it was in first-person (laughs)... it would be kind of amusing... maybe we should do that (smiles) that'd be fun.

At 14:05, responding to another question from chat:

> Do I find the language scaling, well, now that I am dealing with a more complex program?

> Yes. I've been dealing with a program this complicated for like five or six months so this is nothing new.

At 14:15, responding to another question from chat:

> Do I have my own shader language?

> No, right now we are using OpenGL and [unintelligible] GLSL.

The answer to this third question indicates to me that he is focusing on getting something that works better for him than pre-existing alternatives without getting bogged down too much that every side of his language needs to be perfect right off the bat.

In other words, he is balancing the time to perfect his language against the time he wants to spend making games rather than spending 100% of his time working on the tooling and not getting to write any games.

And while we're on the subject of people making their own languages to make games, see also Andy Gavin.

He made Game Oriented Object Lisp (GOOL) for himself and the other people at Naughty Dog to use when making Crash Bandicoot for the original PlayStation.

He later made another language, Game Oriented Assembly Lisp (GOAL), for the Naughty Dog game Jak and Daxter: The Precursor Legacy for the PS2.

https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp

He has written a series of articles about the making of Crash Bandicoot on his website. Well worth a read!

https://all-things-andy-gavin.com/video-games/making-crash/

You talk as if this was an example of a successful strategy of writing a general purpose language for one program.

However Jon Blow's successful games, Braid and the Witness, were not written with that new language.

I am just saying it seems like a good strategy. The mention of Braid was just to give some context about who I was talking about since I think at least more people have heard of Braid than the amount of people who knows who he is if you don’t state that he made Braid.

He doesn’t have a programming language. He has a series of YT videos.

Until he open sources the code (or at least lets someone else try it/look at it) discussing his “language” is giving him too much credit.

Can you quote any project were this strategy was commercially successful?

UNIX/C, for one :-)

> I believe the world would be more interesting if more people applied it—because then we'd get more qualitatively different new systems.

Interesting, yes. More productive, not sure.

It's basically quality versus quantity, or in the words of Stalin: "quantity has a quality all its own".

So far scaling horizontally (more people working together) seems to fare far better than faring vertically (one or a few people being way more productive alone or in a small group).

Reply on news.ycombinator.com