The Quorum Programming Language

In other words, it wasn't the syntax, but the concept of a program running from top down (including loops or conditionals), how expressions work, and what a variable is and means. These fundamental concepts are sooo far beyond the comprehension of people it took a LOT to get down. After that stuff was understood, learning curly braces, keywords or indenting (for style not meaning) was so damn easy for people.

So I would expect an evidence based programming language to look completely different than this.

Edit: It looks like a primary use-case for this language is for blind people - for which use-case I know nothing about. It wasn't clear from the Quorum language website, but it is clear from the video linked by another commenter.

That definitely would affect how I would have written my comment if at all. Take my original comment with a grain of salt that may not even be relevant here.

It's especially galling to call out the research of the space as largely absent and then hang their hats on a narrow set of publications. If the space is not well-researched, any divination from the current body of research will, most likely, be deeply suboptimal. You can't claim to be "evidence-oriented" in the absence of evidence.

I get that they're trying to spur the K-12 space, and that's arguably admirable. Still, I don't expect this to become the next big thing, even for a few over-zealous school teachers. Kids will continue to use python, processing, sketch, C++, etc... Pull them in with the goal (e.g. FIRST robotics) rather than trying to slide them in by promoting a toy language to 14 year olds.

Maybe I'm old, crusty, and wrong. We'll see, but I wouldn't be at all surprised if this is the last time I hear about Quorum.

First, people saying Quorum was originally designed for blind children are correct. Originally, we were toying with the idea of adjusting syntax to make it easier to read out loud through a screen reader. That stemmed from observing young blind programmers use a variety of programming languages on one of my first National Science Foundation projects. Phrases like, “for int I equals semicolon, I less then semicolon I plus plus” or other phrases can be extraordinarily difficult to listen to, especially if you are child. This is one reason why Quorum is used so heavily at schools for the blind or visually impaired in the U.S. and elsewhere. We haven’t been just for the blind in a long-time, but I still care about that community quite a bit and we work hard to make things accessible. There’s a ton more to discuss in that area, but I want to move on to “evidence."

Second, I think some folks don’t quite understand or have context for what evidence based means. For those just learning about that idea, I would recommend reading Antti-Juhani Kaijanaho’s dissertation on the topic, especially later chapters like 8. That can be found here: https://jyx.jyu.fi/dspace/handle/123456789/47698. There’s few points to consider, especially 1) the amount of human-factors evidence that exists in the literature is very low — it is a fact. It’s an uncomfortable thing to say that the research community didn’t gather evidence, but it’s the truth nonetheless, 2) the methodologies used by research scholars in the literature often lack even basic checks and balances (e.g., control groups), and 3) research scholars are still hammering out what evidence based means for programming language design. Antti has his take on it, but there’s plenty of scholarly debate on the issue. The Quorum project, not just the language, aims to both create a language that improves with evidence over time and and also contributes to the literature on what that means.

Similarly, I put together an international session on the topic just a few months ago. The report from that can be found here: http://drops.dagstuhl.de/opus/volltexte/2018/9288/ There is a lot to digest in that report, but the short and skinny is that there are a lot of problems in language design and computer science right now. For example, I argued heavily that computer science needs an “evidence standard” like exists in other fields (e.g., CONSORT, WWC), but we aren’t there yet. For those of us wanting to create tools or products where design decisions are traceable back to evidence, that’s tricky, because structured reviews and meta-analysis are hard to create in a systematic way right now. There’s a million more things to say about evidence, and in fact I’m writing a book on that right now with my good friend Stefan Hanenberg in Germany, but I’ll move on to Quorum a bit.

As a final thing, one common question I get in regard to Quorum is similar to what one comment below said, which is usually something like, “But what if the evidence shows you were wrong and you have to change the language?” As it happens, this is a great question, but has a rather boring answer. First, because have rigid requirements about what evidence means (e.g., basic RCT design), science moves slowly. Quorum’s core syntax and semantics change slowly. If anything, I suspect it’s a little slower in the pace of change than other languages. Second, not every decision in Quorum is based on evidence. The larger an impact of a design change, the more evidence we require. If someone alters a not-often used API, or adds new functionality to the standard library, that’s generally fine if it passes our internal technical and peer reviews (which check against known evidence). If, instead, someone wants us to add to core syntax and semantics, then the evidence required is considerable because whatever our user count is today (I think around 45k people, which if I recall has ballpark doubled year on year).

Naturally, however, this doesn’t fully answer the question, because sometimes studies really do show we got something wrong. For example, Brett Becker (he’s a fantastic scholar in Ireland) recently did some work on compiler error design and Quorum’s design needed to change to conform. As such, we’re working with that group to make adjustments, the first of which came out in Quorum 6. In that case, it’s relatively easy to make the changes because the work suggested we should change the compiler error messages. For more core changes, like syntax, it changes rarely, but when it does, those changes come out once per year, which our user community is well aware of and we tie to K-12 school schedules to not screw up teachers. When that happens, our user community can actually see the evidence we see. Many programming languages change their designs over time, but in Quorum we 1) gather evidence before doing that, 2) often put our evidence through the scientific peer review process, and then 3) change everything on a cycle that our users know about.

Anyway, so there you go. If nothing else, I hope it provides people some context. Hopefully, when our book comes out (TBD), there will be lots and lots more information.

https://www.youtube.com/watch?v=uEFrE6cgVNY

Evidence-Oriented Programming" by Andreas Stefik 2015