Why Crystal is the most promising programming language of 2018
medium.com
medium.com
But I'm a bit vary as for the casual observer the Crystal project github (https://github.com/crystal-lang/crystal) seems brim with unsolved issues (over 500 issues!) and stale PR-s (over 105!). I'm not sure if this is a sign of poor project managment, development stall or poor community engagement but still it leaves a bad first impression. Contrast this to elixir project page (https://github.com/elixir-lang/elixir).
Maybe someone who is actively involved in the development of Crystal could shine a light?
Keep up the good work!
Some projects decide to hide those by closing the issues, we generally keep them open, because they are still valid issues.
You missed another possibility: it being perfectly normal.
If you check any large-ish project you'll find 100s or 1000s of unsolved issues and stale PRs.
And I'm talking about seasoned, used in production, products, from MySQL to Chromium and whatever...
But don’t get me wrong, I <3 Ruby and am excited about Crystal.
OSS doesn't care about lean/agile, those are ideologies consultants sell to enterprises.
Things are done when they are done.
Totally depends on the nature of the project, its policies and users.
Any popular FOSS project that allows anyone to post a bug, and doesn't just close unattended bug reports (which is bad imho), is likely to end up with some (usually minor) bugs open over 12 or more months.
Something I've noticed when reading 'X language is great' posts, the comparison is usually limited to a narrow subset of what's out there.
As an example, I think D meets all of the above criteria. Maybe I'll give on Ruby-like syntax, which D instead has more of a C-like syntax with adjustments to improve compile time.
There are lots of great languages out there and it really irritates me when I see claims of 'X is better than Y or Z' and it seems no effort is made to see what else already exists with the same (or greater) set of features.
However for production code I only use the programming languages directly supported by platform owners.
Of course others rather take part in ramping up eco-systems and that is fine as well, otherwise we wouldn't get new toys adopted by platform owners.
Out of all of the languages I've read about the past few years, only 3 jumped out as bringing real value to the table compared to the other options that are already out there: Go, Rust and Elixir.
Fundamentally though, Erlang/Elixir is a language built from first principles to achieve reliability in concurrent, distributed contexts. It's quite unique in that regard. Even the language it's most directly compared to (Go, in having go channels that are sort of but not quite like Erlang/Elixir processes) is built instead from concurrency first, not reliability (and having concurrency flow out from that as a required feature).
So other that the gem-like package ecosystem and a syntax familiar to Ruby developers, it isn't particularly new.
"Is FFI cheap on all interesting platforms?" is a better question, than "Does it compile to CPU-native binary on all interesting platforms?". In this space you only kinda get some JVM languages, but on Windows you would have to do something more special. Scheme to some extent. Also Haxe.
If you don't care about it you can read the rest of this thread with all other favorite languages. I for one cheer for Myrddin, Pony and Zig.
I was thinking lately about an imperative language, that would be easily compiled to other first class languages. To serve as a cross-platform core of application. It would be quite limited - for example no heap allocations. Something maybe akin to Google's Wuffs language. Just an idea on the one edge of the problem.
Once it has that, I'll be happy to learn it, but currently it's concurrency seems to be as limited as Racket's. That's a big obstacle for early adoption, since the trend is going towards 64 cores sooner than later. My current CPU already has 8 physical cores and 16 logical cores, and I'd like to use all of them.
This is not a critique on Crystal or Racket, I understand perfectly well the challenges that parallelism poses to language implementers. But it's definitely a massive shortcoming.
Yeah this is a deal-breaker. And it's kinda weird that the article is pitching Crystal as a replacement for Go, Elixir, and Rust with that kind of limitation.
I guess this is just the ramp-up into one more trip around the giant ferris-wheel that is language popularity cycles.
1. via `places` - basically spawning a new OS-level process and communicating with it. Has an advantage of being able to talk to processes on other machines.
2. via `futures` - OS-level threads, which insert synchronization points automatically where necessary. You have to structure your code in a particular way to make them not to sync, at which point you get a safe parallelism within a process. You get a future visualizer which helps in finding unsafe (syncing) memory accesses.
So the support for parallelism may be nowhere near Erlang or Pony, a bit worse than Java and a bit better than Python or Ruby.
Before you criticize me for this statement, please let me assure you that it's based on plenty of experience. I'm using Racket for almost all my programming since Racket 2 and have contributed libraries.
My hope is that they add good parallelism support in Racket 7, and since it's based on Chez scheme all thread support should already be built in. Places suck.
Could you specify what are the differences from Go, in this regard are, please?
Nim also supports lighter-weight parallelism with the `spawn` (unsafe, unstructured parallelism which schedules a procedure to run in a thread pool) and `parallel` (lets you use a subset of the Nim language to create concurrency safe tasks) keywords which pass tasks to thread pools in the background. You can read more about Nim's concurrency here: https://nim-lang.org/docs/manual.html#parallel-spawn-spawn-s...
Go has Goroutines which use less resources than threads on most operating systems while also taking advantage of multicore CPUs (I assume they must run in a thread pool running in the background?). It also supports a select case statement which is a nice bit of syntax sugar for managing multiple channels, which Nim doesn't support (although its macro system is pretty powerful, so you could plausibly come up with an equivalent). Go does not support shared memory communication as a language feature, although some libraries apparently give you that option.
First of all, the problem isn't that it's "compiled" - Python is "compiled" too, but still has one of the best REPLs available.
The problem is that the language is static. But static (and compiled) languages can have REPLs, for example C++ has an excellent REPL called Cling.
It's not impossible, just requires some more effort.
As for Crystal, it looks like a neat language. However, if it's going to try to compete with the fast compiled runtimes such as Java, it will need some parallelism support.
In a way it is. For a REPL you most likely need an interpreter for your language, like Cling is for C++.
Isn't that another way to say "interpreter"?
Meteoric! Because we all know the the Tiobe index is incredibly accurate.
Would have been nice to see some code examples, rather than just handwavey bullets points that you could write about nearly every up-and-coming programming language.
Meteoros is Latin and means "to raise up", usually into the sky. Hence "meteorology" which is the study of the sky and "meteors" which are bright lights in the sky.
People don't say "meteorological rise" do they? They're not talking about gradual lifting, or about eyes. They're talking about a meteor. That's a real English word for a real physical thing, which by definition is approaching the earth - i.e. falling. Even if there's some etymological relationship to rising, the dominant relationship between the metaphor and the thing to which it refers in practically any listener's mind is all screwed up. Learn a better phrase, or even better make one up, instead of rationalizing one that's broken.
More importantly, the TIOBE index is just a web noise indicator. A dedicated marketing guy focused on dumping the project's keywords throughout the WWW is all it takes to manipulate the ranking.
A difficult implementation detail, but it shouldn't affect the core of the language.
"Parallelism first"? That's a contentless statement that borders on marketing speak...
This has nothing to do with being a "compiled" language, nor with being a statically types language for that matter.
If languages like Haskell and Idris can have a repl, very basic languages like Crystal certainly can. I can imagine it not being their top priority though.
Then I read the comments and the largest single complaint is lack of parallelism, by which people seem to mean being able to utilize all the cores of the processor.
I've been programming for a while, but haven't run into this issue.
Can someone explain who uses parallelism, for what, and in what context?
In principle, I know how it ought to work. But when would I need to use it?
For instance, if someone is making a web api, and you're pulling data from a database, should one think about it?
If one is doing some data analysis, should one think about it?
I always thought that it is a "low level" program that takes care of it. So tensorflow might worry about it, but I wouldn't if I use tensorflow. Or perhaps mapreduce worries about it, or the ORM worries about it.
But when should I worry about it?
You should worry about it when you want to do something faster than you are already or want to do more things at once. Maybe someone's framework/service has done it for you maybe not.
For a web api, the workload _usually_ looks like: {database call} -> {do some work} -> {maybe more calls} -> {more local work} -> {done}.
In the python/node/ruby world where you have mostly single process workers. You get some parallelism by doing work locally while the database call is made asynchronously. If you want to handle more requests on a host you'll just run more processes and load balance across the front (nginx, gunicorn, etc). In languages with threads you usually do that INSIDE your program. A single process load balances across worker threads internally.
At first blush, multiple threads are pretty similar to just running multiple programs. It gets more complicated when you start talking about how you want to communicate between them or share resources across them.
For data analysis you often have algorithms that are embarrassingly parallel[0] chained together. You use threads the same way you'd call three friends to help you move. You all pack boxes at the same time without really needing to coordinate and then you start loading the truck where you have to talk more. One might wait in the truck and organize while 2 or 3 others might be shuttling boxes outside. Maybe everyone sits around while the stairs are blocked by one person moving a shelf. It just depends. Having 4 people helps sometimes or is disruptive in others.
[0]: https://en.wikipedia.org/wiki/Embarrassingly_parallel
P.S. - To address your examples: tensorflow, map-reduce, orms. etc.
It's all still about parallelism but not necessarily about threads.
Parallelism is just doing more things at the same time. Threads are (mostly) about CPU parallelism, but many of those use other resources to get parallelism. Tensorflow might use the GPU, which has many threads internally (simplification), to do one thing while the CPU does another. Mapreduce will use a lot of different machines across a cluster, but maybe only a thread or two on each, to archive parallelism (and really getting disk IO parallelism). And ORM will call the database and let it do work while your CPU else instead of wait.
This was the best part if you want to expand on it in a longer post: "You should worry about it when you want to do something faster than you are already or want to do more things at once. Maybe someone's framework/service done it for you maybe not.
For a web api, the workload _usually_ looks like: {database call} -> {do some work} -> {maybe more calls} -> {more local work} -> {done}.
In the python/node/ruby world where you have mostly single process workers. You get some parallelism by doing work locally while the database call is made asynchronously. If you want to handle more requests on a host you'll just run more processes and load balance across the front (nginx, gunicorn, etc). In languages with threads you usually do that INSIDE your program. A single process load balances across worker threads internally.
At first blush, multiple threads are pretty similar to just running multiple programs. It gets more complicated when you start talking about how you want to communicate between them or share resources across them."
A few questions:
Out of curiosity, is there a company behind Crystal? or is it a BDFL situation? The press on Crystal has been pretty significant lately which is great. I ask mostly out of historical interest. IIRC Rust/Go seem to had company backing while gaining traction but Python/Ruby were more organic BDFL led projects. Naturally C/Java/C# were literal company products.
While the Ruby-like syntax is not to my taste (prefer Python) it got me thinking about what are the big syntax families? Off the top of my head I count: Lisp, C, Python, Ruby, Prolog, ML, APL? Preemptively apologizing for only knowing the popularizers over the originals.
I read through the Crystal language docs (meaning just the syntax etc) and as a seasoned C++ and Python developer who is _constantly_ looking for something with better performance (than Python) yet much cleaner (than C++), I think Crystal has a lot going for it so far.
Any "great" language should be able to take a thought in a developer's head and easily allow 1) the concise expression of that thought, and 2) efficient evaluation of that thought. I mean, those things we probably would all agree on.
It saddens me to say this, but C++ is falling over as a result of it's own weight. It's become a language for experts. Perhaps more than any popular language, it can take a simple idea in a developer's head and turn it into pages of code. It's actually quite embarrassing. I won't go into that further; judge for yourself (and sorry if that comment offends anyone -- I love C++ and use it every day). But you just can't beat the speed... Well-crafted C++ _should_ exceed the speed of even C (Why? Because templates...). As an aside, it's disingenuous to put Java in the same speed category as C/C++... The "fast" Java programs out there are basically C with Java wrappers (ducks thrown tomatoes). And just like C++, Java is very noisy (but for different reasons).
Python as we all know sort of takes the opposite approach, with dynamic typing a design-as-you-go mentality. And boy what a success it has been, with a flourishing package ecosystem. There's lots of good things to say about Python, but it's f*cking slow as hell (Cython is a hack, Numba shows promise, but PyPy isn't much faster... I was excited about Pyston but don't know where that went). It's not the fault of Python that it's slow -- it's the price of such a wonderfully dynamic language.
So enter things like Crystal. And trust me -- it's definitely early days with this language. But I like the fact that the designers really seem to care about the things that (to most of us I think) matter.... Taking an idea in our brain and putting it (simply) down in code, and then having that code run quickly. Yay!
In this day and age where we are swamped with hype from all of these new languages, let's give praise where it's warranted -- to the people are out there that are trying to refine decades worth of thought and finally "get it right".
My hat is off to those people out there that are forging ahead with these types of projects. Don't mind the criticism -- keep it up and great job.
Sample factorial function:
(define fac (lambda (n) (if (= n 0) 1 (* n (fac (- n 1)))))) ; <<--- look at those parens!
And with Clojure, you don't have proper tail recursion, so you'll have to add some Clojure-only thing in there to prevent the above code from blowing up for large numbers.
There's really no getting around it -- Lisp, for many people, is just hard to parse. Consider:
def fac(n): n < 2 ? n : n * fac(n - 1)
Yay! No noise. But I agree, there are some cool things about Clojure.
what I dream of is a Lisp with an additional mechanism to define precedence of functions/operators. And infix operators. Almost like Haskell.
Read syntax:
https://en.wikipedia.org/wiki/CGOL [1973!]
Macros:
https://stackoverflow.com/questions/25434538/define-function...
def fac(n): n < 2 ? n : n * fac(n - 1)
def fac(n): n < 2 ? n : n * fac(n - 1)
+ more stuff
One level of parentheses comes from using a dialect which uses a variable definition and lambda to define a function, instead of having function-defining syntax like defun.
n < 2 ? n : n * fac(n - 1)
You've loaded the readability dice by writing it on one line. To help with the readability issues in both languages, we can use multiple lines and indentation:
def fac(n):
n < 2
? n
: n * fac(n - 1)
(defun fac (n)
(if (< n 2)
n
(* n (fac (- n 1)))))I fail to imagine a kind of application that would warrant to learn a new general purpose language on top of what I already know. Well, except massive parallelism for which I would probably reach for Erlang or Go.
Impossible, or just hard?
I didn't realise that SBCL is interpreted.
So compiled code can have a REPL, but a REPL needs an interpreter as well.
OTOH LispWorks uses an interpreter for the REPL interaction.
Yes, an Interpreter offers some more interactive features - but usually lacks the compile-time warnings/errors.
Now think about this in the context of a REPL, which again stands for Read, Evaluate, Print, Loop. If you write code and evaluate it right away, this whole business about checking your work won't work right. You would basically have to recompile your entire program on every REPL eval, which defeats the whole purpose of a REPL. There are lots of other tricks to getting this to work, as the article hints at, but they're all very unsatisfactory in delivering the freedom of a dynamic language.
SBCL:
* (defun foo (a) (+ a 42))
FOO
* (defun bar (b) (foo 10 20))
; in: DEFUN BAR
; (FOO 10 20)
;
; caught STYLE-WARNING:
; The function was called with two arguments, but wants exactly one.
; (SB-INT:NAMED-LAMBDA BAR
; (B)
; (BLOCK BAR (FOO 10 20)))
;
; caught STYLE-WARNING:
; The variable B is defined but never used.
;
; compilation unit finished
; caught 2 STYLE-WARNING conditions
BARLanguages like C which require header files and a pre-processor are much harder to write REPLs for. You can try to do it and you can build a thing that works kind of like a REPL (look I type code and press enter and it evaluates, yay), but the experience always falls short of using a dynamic language in a REPL. It's just not even close, and that's just due to the simple fact you're trying to use something in a way it's not intended. Compiled languages simply are not designed with a tight development loop as a major design goal.
I would question what is meant by a 'true REPL'. It's certainly seems possible to build a REPL of a statically typed compiled language.
I’m hoping someone eventually makes a new one, possibly based on MIRI.