Designing a functional programming language: Yatta | Hacker News Reader

Designing a functional programming language: Yatta

functional.blog

91 pointstodsacerdoti44 comments

Open articleView on HN

FWIW, Frank Atanassow, "Some words of advice on language design"

http://lambda-the-ultimate.org/node/687#comment-18074

> Before you go off inventing new programming languages, please ask yourself these questions:

> 1. What problem does this language solve? How can I make it precise?

> 2. How can I show it solves this problem? How can I make it precise?

> 3. Is there another solution? Do other languages solve this problem? How? What are the advantages of my solution? of their solution? What are the disadvantages of my solution? of their solution?

> 4. How can I show that my solution cannot be expressed in some other language? That is, what is the unique property of my language which is lacking in others which enables a solution?

> 5. What parts of my language are essential to that unique property?

All very good questions. However there is missing one question, that perhaps overrides them all:

- By doing this will I gain a deeper appreciation of the art? Will it help others to see this too?

Designing a good language is very very hard, harder maybe than a typeface, or a cathedral. But the rewards, in the terms of that question, can be inestimable.

There is a lot of truth to this. I'm in the process of designing a language with integrated SQL where I "code-generate" the in-memory tables, and beyond appreciating language I also appreciate relational tables for what they do.

What I am finding really nice is that I don't even have maps in my language since I can just use tables and get the benefits of maps along with indexing and all the fancy stuff.

This post has always irked me. It reads as straight up gate-keeping, especially when what follows these questions are a litany of negativity ("go home" "no one cares" "you are kidding yourself" and "you are wrong."). This is especially frustrating as the author later admits he himself cannot answer these questions honestly and yet persists in designing his own language. He certainly hasn't "gone home" so why should anyone else?

Here's my advice: Want to invent a new programming language? Dive in. It doesn't matter if someone invented it before. It doesn't matter if you can't formalize it in mathematical notation at the outset (or ever). It doesn't matter if your language could be implemented in a library of another language. It doesn't matter if you can't articulate the precise benefits of your language. It doesn't matter if there are any benefits at all. It doesn't matter if you can't answer a single of the above questions. Have at it. You are guaranteed to a) learn a lot, and b) have fun. Maybe you do something completely derivative and that's fine. Maybe no one will appreciate what you do and that's fine. Maybe it will go nowhere, maybe it will never compile. Maybe you will be the only user. Maybe some guy on the internet will tell you to go home and no one cares. So what?

I don't see it that way. There are thousands of languages now. It's wise counsel to people (like me!) who hope to introduce a language that actually gets used. If you can't "check those boxes" on this handy checklist, well, maybe you gotta spend more time at the drawing board.

For those of us that are going to do it anyway, with no expectation of becoming popular and widely-used, I would treat it a stimulating challenge rather than a slammed door.

I think there's something to be said for going through the exercise even if the answer to every question actually does indicate the language shouldn't be written. What happens when you come up with an idea that actually does have a place? It's better to start with the background of having done it before than try to create a dream project with no experience in making some toys.

I can see both points of view. Of course, if you enjoy making a new language do it, lots will be learned and you can always remember back to that language you made.

On the other hand, if your intention is to make a new language that does something distinctly different and for some purposes better, then answering those questions will save you from spending time on things that don't deeply matter. Now if we're not in a circle of language design researchers, things that matter can be syntax.

I can see what's being attempted and I appreciate it. I myself have wanted a kind of 'referential transparency' when it comes to lazy values in an otherwise strict language. Or in this case async ones executing concurrently. You can use the result of the expression without the ceremony. I'm not particular to the syntax and would have liked to see it added to a popular language. Making a whole new language is scratching an itch, which is also great for what it is.

I wish I had been given this advice many years ago, but I got some negativity that trigger a cascade of depression ( https://news.ycombinator.com/item?id=226480 ). I was very young, and I didn't ask for feedback. It was not great waking up to a HN post. In a way, I was not prepared to accept honest criticism for something that I was building.

Granted, it was a bad idea to build a non-tech business on a new programming language in a region which could not hire people to fix the language, so it was the right call to give into the depression and move away from my language. However, it probably took me a decade to try again recently.

When I reflect on history and how things could have unfolded, I realize I was onto something and I was at the wrong place to exploit my ideas. As an example, if I had persisted, then my language could have been what Hack & HHVM is today. However, I was in the mid-west operating on my own.

This time, I'm swinging hard with my new language, and I hope to redefine a portion of the computing landscape. Or, at least, improve it within my myopic universe. I haven't published much, but I have a novel language at the moment which is enabling me to do things that are hard to imagine.

I think this is good advice under one assumption: your intent is to create a practical language that other people will adopt for real products.

Reasons you might make a language that isn't bound by the above questions:

- For learning

- For fun

- To explore a concept

- For sheer novelty

- To make a point

- To have a language you like, even if nobody else in the world cares for it

To put an even finer point on it:

- to indulge in syntax design

The author of this advice explicitly calls out syntax design as pointless. As if Java, Lisp, Python, Haskell are the end-all of syntax design. Haskell in particular is notorious for indulgent syntax design.

One thing I've noticed about my students (undergrads) is the first thing they are interested in playing with when it comes to language design is syntax. They have ideas about syntax they want to try out, and it gets them interested in parser design. Probably because it's the most tangible thing they have access to and experience with. When they try out their syntax ideas it opens up new pathways for them and they eventually move on to greener pastures.

But the idea that syntax is done and solved and there is no more room there for innovation is completely foreign to me. What about syntax in other (natural) languages? Languages written from left to right? Top to bottom? Languages with a non-roman character set? What about syntax with emojis? There are entire programming languages written with emojis now (https://esolangs.org/wiki/Emoji). Did the author of this consider them in 2006?

Don't forget languages written in the form of a bitmap :)

https://en.wikipedia.org/wiki/Esoteric_programming_language#...

"for learning" + "to explore a concept"

-> to understand why things are (or were) done a certain way

-> to test the boundaries of what's possible

Surely no one thinks that current languages are the most efficient or effective possible way of doing things? That we've even come close to enumerating all useful methods of expressing program execution? (Take a look at what Racket is doing for example.)

You're preaching to the choir my friend. (^_^)

Here's mine: https://joypy.osdn.io

> Thun is dialect of Joy written in Python.

> Joy is a programming language created by Manfred von Thun that is easy to use and understand and has many other nice properties. This Python package implements an interpreter for a dialect of Joy that attempts to stay very close to the spirit of Joy but does not precisely match the behaviour of the original version(s) written in C. The main difference between Thun and the originals, other than being written in Python, is that it works by the “Continuation-Passing Style”.

I don't know if I wholly agree.

These are good questions with regards to language design.

But for some people there is a massive learning experience that comes from designing and implementing a programming language and it's compiler even if it never becomes a successful or popular language.

I feel this learning experience is has plenty of value even if the end resulting language does not.

Anders Heljiberg wrote Turbo Pascal, J++, and Delphi, before moving on to things more widely accepted as successful like C# and Typescript. Yes Turbo Pascal and Delphi had some large followings

Likely his experience from previous programming languages allowed to him be successful later.

> I do not want to fall in the trap of slow compilations and debugging mysterious typing errors again

I always found that, instead of debugging type errors, in dynamic languages you end up debugging runtime errors.

The errors don't go away, but with compiled languages at least they're easier to catch.

I'd add to that that type errors can happen at run-time, too, in any language that allows any level of dynamic typing. That includes fully dynamic languages (I swear, the most mysterious type errors I've ever fought with concerned Python libraries with insufficient documentation), but it also includes static languages like Java that permit unsafe downcasting.

It appears to me that the main selling point here is automatic parallelization, with futures/promises being abstracted away.

Agree:

- Dead-simple concurrency model. Even if it won’t be the most performing one for some (or most) situations, it should be built into the language and be very easy to use. Ideally built with functions alone, requiring no special syntactic constructs. Advanced concurrency possible via STM.

Most of the other design considerations sound like a description of Clojure:

- Plain and simple syntax, minimal boilerplate, few keywords, no need for OOP constructs. The code must be very easy to read through. (this one is subjective I suppose. :-))

- Everything is an expression, no statements. The program itself is just a single expression.

- First-class functions and modules. Modules are simply sets of functions and records. Modules can be further organized into packages. Function calls supporting tail-call optimizations.

- Few data types. Yatta comes with the following built-in types: integer (64 bit), floating point number (64 bit), character (UTF-8), symbol, sequence (variable length, constant-time access to both ends), tuple (fixed length), dictionary, set and STM (Software Transactional Memory – coming in beta). No implicit conversions between types though!

- Custom data types, or records. Records are basically just tuples with named fields, and syntax sugar that goes with it.

- Run on JVM (GraalVM) and allow polyglot cooperation with other JVM languages. Very important if integrating with existing code-bases.

Yatta: It's so easy, happy go lucky! https://youtu.be/rW6M8D41ZWU

I would like the Elixir without the Erlang.

I would like the Python with some pipe forwarding.

I would like the Python as a binary.

I would like the package management of Rust.

The speed and flexibility of C, but more consistent, and with less ability to make mistakes. Allow for a power mode, that turns off bounds checking, to boost speed.

I would like a language that allows seamless piping to and from programs, specifically Unix tools. Just now my favourite language is Bash because:

- concurrency is so simple that you forget you are doing it,

- I can easily use other tools that far smarter people have written.

The ability to execute other programs and capture their output always seems to be an afterthought in most languages.

I agree, I want that too. It was not an afterthought in case of Yatta but rather a difficult tradeoff to make in terms of extending the syntax (which I try to keep minimalistic) and providing an easy interface for interacting with external processes.

You can see the result of making this tradeoff here: https://yatta-lang.org/docs/stdlib/system/#running-a-pipelin...

Not as "seamless" as bash maybe, and perhaps not feature-complete yet, but I hope this can serve as a starting point and we can build on this for future releases.

Brilliant!

I should have read more of your documentation. Which is very readable by the way.

Thanks! Don't hesitate to get in touch in case you encounter anything unclear, wrong, or missing.

There is https://xon.sh/ which calls itself "a superset of Python 3.5+ with additional shell primitives that you are used to from Bash and IPython". The tutorial at https://xon.sh/tutorial.html is fun. I did use it as my primary shell for a while many years ago. I think I stopped because it sometimes got confused about colored terminal output or something like that, it's probably much more mature now.

That also looks interesting.

After akovari and tom_mellior's replies I now have to resist the temptation to rewrite a bunch of scripts in xonsh or yatta or I'll never get them finished.

:-)

I can only encourage you to give it a try and share your experience with us :)

It seems the main point of this language is implicit concurrency. From what I have read, from the link above, the main idea is that it executes individual expression trees in separate threads, taking care of dependencies between threads and handling promises appropriately.

It's a nice idea on paper, but I wonder how soon developing a "real" application with it will result in performance bottlenecks because the compiler could not capture the intentions of the programmer or the program is really difficult to untangle into different threads of execution.

It doesn't parallelize just "any" expression though. Only blocking functions, such as those working with IO. Similarly, in JS for example you would use callbacks to avoid blocking. In Yatta you don't have to explicitly worry about callbacks, Promises or alike, because they are hidden into the runtime itself.

If you want to parallelize a computation, you can use function async, providing it a lambda, and then yes, Yatta can parallelize that as well (given that there are no other dependencies on it). But this is an explicit thing in control of the programmer.

they should have just shared the project landing page => https://yatta-lang.org

why another language? why not, who cares

maybe steal some date flow ideas from Oz

https://en.wikipedia.org/wiki/Oz_(programming_language)

What is it with functional advocates and their obsession with brevity to the point of reductivism, and lack of visual structure

Succinctness is power.

Paul Graham explained this way better than I could: http://www.paulgraham.com/power.html

On the other hand I don't agree with your point about visual structure. Personally I believe visual structure is mostly about code indentation. Visual structure can be apparent in any language, functional or not.

I don't understand. It looks a little like Ruby and Elixir mixed together, maybe without some colons that would otherwise be there. There's nothing crazy going on in the examples in the way of single letter variables or anything else I'd consider reductive.

It's not really all that novel, and more of a pleasing combination of existing ideas. So why pin this on "functional advocates", whatever "this" even is?

Maybe you could expound on your ideas a bit to help me understand your reasoning. Perhaps your comment is just incomplete rather than being wrong.

(also, hopefully your qualms go further in depth than the syntax, because literally-skin-deep comparisons are a little boring)

They're a cult of aesthetics. FP is just one tool in the toolbox. And for most things, it's the wrong tool.

Interesting. I agree that FP does have a bit too much fun with code golf, but I found the code examples in the article to be very clearly structured, readable, and not at all terse.

Is it possible that your perception of a lack of structure is down to a lack of familiarity with the syntax and idioms in ML-style languages? I could see where perhaps not knowing what parts of the code are syntactic and what parts are stylistic causes it all to fade to gray.

I've noticed a phenomenon along these lines when people are learning lisp: non-lispers and lisp learners tend to see nothing but parenthesis, because that's the most visually distinctive feature of lisp code compared to other syntaxes. More experienced lisp programmers, on the other hand, have a tendency to just tune them out. In well-formatted code, the parens themselves have very low information density, so you don't necessarily get a lot of value out of scrutinizing them.

What do you mean? The code example is more verbose than Python, and has about as much structure.

what's the equivalent python? (must include async, and proper error handling over the async connections). I'll give you a break on the "not parallelizing if it's got datapath dependencies".

Elixir for the graalVM

It would be great to start a new programming language by stating what sort of actual problem it will solve (or handle better) than the other gazillion languages. Example: Rust -> memory safety, because based on many studies memory safety is responsible for ~70% of bugs. If you are just doing a language for the fun of it is still ok, even though I would be less interested in reading the post. As far as Yatta goes:

"allows me to write programs as simple expressions, without having to worry about low-level stuff such as concurrency, with a beautiful syntax and a strong but dynamic type system."

The first part is true to I do not know how many languages, the second part is subjective, I think LISP and ML have the best syntax some people throw up after opening the hello world examples.

Yeah, nothing is a bigger turn-off for me on the home page for a language or framework than starting off with "Helps you build [some combination of] simple, maintainable, fast, scalable software". No kidding! You and every single other language out there.

This one isn't nearly as bad as that, but the point remains: tell me up-front why I should care.

> The first part is true to I do not know how many languages, the second part is subjective

And it's fine that way. There are lots of languages out there, and very few of them have any objective problem-solving features to justify their existence. Tackling some problem in a new way like Rust does (arguable, see prior art like Cyclone) is the exception, not the norm. Lots of languages are about improving subjective ergonomy while combining and tweaking features already found in other languages.

Would it be fine to state that the language solves nothing for you and you can stop reading? But I don't think that needs to be stated any more than the post already did.

Then the rest of the readers, who have an ounce of curiosity left in them, can scroll on and take the time to appreciate the design and see if they like where it's headed.

Reply on news.ycombinator.com