Parallelism in Crystal | Hacker News Reader

Parallelism in Crystal

crystal-lang.org

139 pointssdogruyol41 comments

I’ve been using Crystal on-and-off for a couple of years and have started to use it as my primary language for most non-enterprise work. The current web-app I’m building is primarily written in Crystal and it has been an absolute joy to use, and even if I don’t strictly need parallelism in this project I’m glad they’re pushing though with the big language changes.

It’s been so successful at everything I’ve needed it for, I wouldn’t consider going back to either Ruby or similar interpreted languages, regardless whether it’s a little CLI helper or a larger monolith server app. I’ve found my ability to allow the compiler to catch bugs incredibly reliable without feeling bogged down with types. I hadn’t worked much with typed languages previously and as a Ruby developer it allowed me to ‘learn’ types without even feeling like I was doing any learning.

It’s still somewhat of an obscure language so I thought it would be helpful to describe my experiences with it.

Do you use much external libraries because ecosystem I'd pretty much what makes a language useful after the language being solid by itself.

Also, I assume Googleability is pretty much close to zero, you'd have to go to the primary sites (their forum?) to ask questions when stuck?

It's obviously far smaller, but I've had luck so far with everything I need being available. What I've seen, mostly Postgres, Redis, and XML/HTML parsing, have also worked flawlessly so far.

Having performance improvements of 100x in real-world scenarios is just astonishing. Ruby is still my go-to, especially because debugging and console support are so much better. But everything that gives me time to think gets reimplemented.

["crystal lang" <question>] usually works for google.

There's actually a surprising amount available in the stdlib for your typical web project. By way of example, I'd always hated how in Ruby each API wrapper gem would make their own decision on which HTTP client they used, so you could end up pulling in 3 or 4 separate libraries just to make HTTP calls to a 3rd party API. People may disagree but I think this is (was?) largely due to Ruby's Net:HTTP feeling janky to use, at least compared to Faraday and the like. Here, the HTTP modules within the standard library feel right, and I've not needed to reach out for all that much from the ecosystem.

But as a simple example I accept that API calls to third party services is low hanging fruit so in terms of finding drivers for lower level protocols I've not had an issue that I'd class as a blocker. There are times where I've had to extend libraries or write a parser for this-and-that (e.g. PostGIS and Paseto, both of which were unused this time round) but I find I need to do this less and less as time goes on.

You do need to reach into the forums or gitter on rare occasions but I can count on one hand the number of times I've needed to, and it's typically because I was trying to do something unusual and quite low-level on the HTTP stack - as in something I've never had to do in a non-Crystal project, such as sending a specific packet. I've always found help there too and people are happy to engage.

I'm curious as to what web-app you're building with it. Is it available to use?

Not yet but not far off... It's an e-learning platform and student management system, backed by postgres and fronted with React. For the most part it has worked well but I need to do a bit more tightening up before I'm happy releasing it into the wild.

I have been using crystal+ vanilla js for my pet project and the sheer joy i get from using this language is something else.

2019 has been a good year for crystal. Basic multithreading is in. Windows support is making good headway. I am optimistic that a working windows compiler would be out by end of year.

The last major dragon for crystal to slay would be the compilation time. It still feels unreasonably high. Hopefully that would be a major focus of 2020

I'm curious as to what your pet project is/if it's available to online to play with.

The Crystal underbelly is here https://github.com/rishavs/noir

the api is consumed by a vanilla js spa. I talk more about my VanillaJS spa's architecture here https://dev.to/rishavs/making-a-single-page-app-in-ye-good-o...

Delighted with this. I've been working on and off on a shortest-path routing engine in Crystal. Parallelism means I can get performance on a par with C++ even on large graphs, but with the simplicity and readability of Ruby. Full marks to the Crystal devs for tackling the no 1 feature request.

Aside, but is there a good class or resource for learning about parallelism in computing? I'm reading the blog post and vaguely recalling differences between threading and forking. I regularly encounter the same concepts over and over but they haven't been sticking. A hands-on project would be great.

I just learned about Crystal a couple of weeks ago and it's completely subsumed every new personal project I have into its domain. I have long missed Ruby's syntax and human-friendly feel, but began to prefer the speed I could find in other languages like Go.

Until crystal get a REPL I would stick to ruby / rails for POC and MVP, then scaling problem "could" arose, so translate it to amber or lucky (Web Framework in crystal) could solve the issue. About Type and Casting Nil, gem like sorbet should be a good replacement in ruby

Waiting badly for an installable ARM port; small boards would benefit a lot from it. Is this planned in the future?

I get why N=2 is slower than single-threaded, but why is N=4 more than twice as fast as N=2?

If you're looking at the first graph, the explanation is right above - the overhead of synchronization.

That's nonsensical. There is surely at least as much synchronization overhead with 4 threads as with 2.

If the overhead is X, and the amount of work that can be done ignoring it is Y, you have Y - X. If you double the amount of work that can be down, but overhead stays the same you have 2Y - X, which is more than 2(Y - X) = 2Y - 2X. You're over performing doubling by not having to double the synchronization cost.

Not necessarily.

There can be overhead introduced between the "1 thread" and ">1 thread" cases.

From 1 -> 2 threads the overhead is introduced, if the cost is somewhat fixed, the gains from 2 -> 4 can be proportionally larger.

Why Crystal over C++?

For very similar reasons that they choose ruby over C++ ?

Why C++ over assembly?

I’m asking because they presented Crystal as Ruby with OOP and high performance. I’m asking because I’m curious, is it the ruby syntax that much better?

You’ll get different answers depending on who you ask but I think most will agree it is much more accessible to new programmers.

Coming from C (no ++) and Java the first thing I noticed after a month with Ruby (in 2005?) was that I didn't write any for loop. I mean, those useless for (i = 0; i < length (array); i++). They become array.each. Then I loved the blocks and later on the postfix if and unless. It's not perfect, it's got its quirks and metaprogramming sometimes is easier in Python, sometimes not, but overall it's the easiest language I used so far. I never tried Crystal because of the compilation step. I really don't need the extra speed for my sw and no customer asked me Crystal so far. I guess it's going to be a similarly pleasant experience.

> metaprogramming sometimes is easier in Python, sometimes not

Now, I've been exposed to more ruby, and more metaprogramming in ruby, than in python - but this stil surprises/intrigues me.

What kind of metaprogramming do you find easier in python?

If I'm not wrong I can add a methods like this

  def m2():
   print("nuovo")

  object.new_method = m2

instead of redefining the class. It's similar to JavaScript. However I don't have a computer at hand now. I can't check the details and what could be done on Ruby.

True, unfortunately. You can't pass functions around as first class objects, you'll have to use lambdas or procs for that (although even then composition doesn't work like that in Ruby).

(No judgement from me on whether that's better/worse/about equal, just confirming.)

This may be what you're looking form, to add a method to a single instance, although the syntax is clearly not as smooth: https://stackoverflow.com/questions/1887845/add-method-to-an...

OTOH, if your problem is slightly different, I find the syntax to add methods to any class very practical and powerful, with of course the caveat that you can produce a maintainability nightmare if overused:

  irb(main):001:0> a = 1
  => 1
  irb(main):002:0> class Integer
  irb(main):003:1>   def hello
  irb(main):004:2>     puts "world"
  irb(main):005:2>   end
  irb(main):006:1> end
  => :hello
  irb(main):007:0> a.hello
  world
  => nil

That’s a pretty shallow view of easier metaprogramming. Ruby allows for many different ways to metaprogram. You can open up any class at any time and create new methods on the class, you can even add a new method directly on an individual instance of a class without changing any other instantiated objects of the same class. I’m just scratching the surface of what’s available wrt metaprogramming in ruby. The ease of metaprogramming in ruby is specifically why rails was easy to write.

I suppose in ruby you'd re-open Object and add new_method there... (in part by overriding method_missing, maybe).

I wouldn't recommend actually doing that, however.

If actually doing this (adding new method to a class) I think it's quite easy to re-open a class in ruby (see sibling comment(s)).

I can see adding methods actual instances (only) might be more work in ruby though.

What would be the typical use case for a special instance in a given scope, as opposed to a special class in a given scope?

Ed: i see qquark's so link address the technical part: define_singleton_method (so no need to open Object to add that...)

Because you are tying yourself to a specific ISA. This is bad even within the same chip family. If you want to migrate your x86 32 bit code to 64 bit then you have to rewrite everything.

Because it's a much saner, coherent, and easier to use language with less footguns...

Now, why not Crystal over C++? Because it's way less mature, with way less documentation, way less resources, way worse tooling, way less libraries, way more implementation bugs, and, if we're realistic, it might never go anywhere...

> Because it's a much saner, coherent, and easier to use language with less footguns...

Which for parallelism probably switches the question to “why Crystal over Pony”, since Crystal seems to preserve the key C++ footguns specifically linked to parallelism.

> and, if we're realistic, it might never go anywhere...

I don’t see why this should be the case, unless we as a community decides it will not go anywhere.

Most languages never get anywhere.

Languages of the quality of Smalltalk, and Ada, and Dylan, and Self, and CL have vanquished in the margins, and e.g. even something as celebrated as Haskell does hardly better in the industry (some financial companies, this or that project, and so on), so the chances of a new language like Crystal are slim to begin with...

Oh yes, been there, done that, created proglangs, and they certainly did not get anywhere. But I feel Crystal has already gained such momentum that it indeed will get somewhere, it’s beyond that phase "to begin with." After all, if any language can get anywhere, why not Crystal?

>But I feel Crystal has already gained such momentum that it indeed will get somewhere, it’s beyond that phase "to begin with."

Let's hope so, but I fear that's wishful thinking. What proof do we have of that?

It's not used anywhere important ("killer app"), barely makes the rest of the TIOBE index, along with staples like NATURAL, Alice, BBC Basic, Euphoria, Factor, Forth, and Icon, and has no major force like Google or Mozilla backing it.

It's only us here in HN who keep hearing about it.

Julia has had tons more publicity (even from some major outlets), and it's still going nowhere fast.

> It's only us here in HN who keep hearing about it.

Don't you think us here at HN include quite a number of people standing centrally when it comes to "anywhere" in the tech industry? This is what I mean, when talking about "us as a community." Of course this does not go for everyone... but for a considerable amount, I'd say. (Myself am certainly not one of them.)

> Julia has had tons more publicity (even from some major outlets), and it's still going nowhere fast.

Well, Julia being made for extensive numerical computations and the like, I think it is doing very well, esp. in academical use, which is somewhere it's right at home. If not yet very much in use yet, at least many academics in relevant fields--which frankly are quite a lot days--have heard of the language. If Julia holds enough of its' promises, I don't see why they wouldn't migrate eventually, given time.

I don't see how any lang could get anywhere "fast," no matter how superb it'd be, "anywhere" tends to be sceptical of change if they're fine with what they have.

To me is just Python-like (or actually Ruby-like but close enough) but with much better performance. I still haven't learnt to use it well though.

It's a joy to write due to its syntax.

Reply on news.ycombinator.com