The sky is not falling – about the supposed end of dynamic languages
yogthos.net
yogthos.net
"I am thinking of a data structure for managing collections of objects. It provides O(1) insert and update operations. It has native hardware support on all modern platforms. It has a long history of use. It's proven, and it's lightning fast.
Unfortunately, support for it in my favourite language, Haskell, appears to be somewhat lacking. There are people that would tell me that it's not needed, that there are other options, that it's bad for parallelism and bad for computer science in general. They say that without it, programs are easier to understand and reason about. Yet, it seems that every time I start to write a program I find myself wanting for it. It's called the store.
The mutable store, that is. I want for real destructive update in a real functional language. I wanted it for long enough that I decided I should take a PhD position and spend the next several years of my life trying to get it."
http://benl.ouroborus.net/papers/thesis/lippmeier-impure-wor...
While I admire him, I think the better solution is to give up Haskell.
https://hackage.haskell.org/package/base-4.8.1.0/docs/Data-I...
https://hackage.haskell.org/package/base-4.7.0.2/docs/Data-S...
"Representing effects in value types is a double edged sword (p. 42). Haskell has fractured into monadic and non-monadic sub-languages (p. 44). Monad transformers produce a layered structure (p. 45)." These are section titles, be sure to read the entire sections which explain in details what the title means.
Ben Lippmeier makes a very good point and is being quoted in full here. Haskell does have issues, and people are trying to fix those issues. If they can be resolved isn't known yet, and the resulting language may be different from Haskell. Haskell is a long way from some pinnacle of perfection, and it is very much not a silver bullet to solving problems. Not that anything in computer software is a silver bullet [1].
For many years I have had a lot of fun with Haskell, doing some NLP, and web stuff with Yesod and Scotty.
However, I am so much more production in a concise dynamic language like Ruby that I am re-evaluating my fondness for Haskell. Also, to a small degree, good IDEs like RubyMine catch some simple errors for you.
I, like Elben Shira, hope for radical new technologies that make me feel like I am developing in Ruby with the safety and speed of Haskell. I am looking forward to the future.
Anyway, code that patches immutable data compiled by GHC behaves much like the plain mutable storage you'd write in C. It may not be always simpler to write (but for complex data, it normally is - and why are you writing so much simple data anyway? is it because of your language limitations?) but is checked by the compiler, what's a great bonus.
Given experienced and disciplined programmers, the type system probably isn't as important. But given the quality of programmers who are using these languages in the real world, a good type system does seem to help.
My personal experience is that Haskell isn't really all that complex - when you stick to the core, it's actually fairly simple, with a small set of well thought out features (compared to Scala, for example, which I love but has a lot of features). Strangely enough, what I find hard when dealing with Haskell is the syntax, because it's so different from all the languages I've been taught or taught myself in the past. That shouldn't be a problem for people that have little to no previous experience and tackle it with a virgin mind.
https://www.reddit.com/r/haskell/comments/3pfg7x/either_and_...
> this experience has taught me that whenever functional programmers claim that certain things are only unintuitive because programmers have had their minds polluted by non-Haskell programming languages... they are usually wrong.
> the more common claims you see about how purely functional programming is easier and more intuitive for students without previous imperative programming languages. Surprisingly, that's not the case. Even in situations where I don't expect it, I'm constantly fighting everyone's urge to think about situations first and foremost in imperative terms.
I think it's safer to assume that functional approaches have inherent complexity compared to imperative approaches. YMMV, of course.
Complexity and intuition aren't the same thing. I would argue that the point of functional programming is to reduce and control complexity.
This is an area where the state-of-the-art in static languages (Haskell, Idris, Agda) falls flat, and I suspect that's why (other than ignorance) we see so many comparisons from Haskell advocates between Haskell and the likes of Python, Ruby, or worse that try to paint dynamic languages as offering only brevity through the omission of type signatures as a major advantage over common static languages--an advantage for which Haskell can largely compensate--while completely ignoring the capabilities more sophisticated dynamic languages offer.
Example of such a language is Java. It's runtime is highly dynamic. Class loaders allow you to create new classes, pre-process bytecode, and many other stuff.
This is a formal way to do homoiconicity, and frankly much better than codewalkers and CPS-conversions and similar tricks.
I now think that everything you can do with macros in CL, you can do in Haskell too and it will be cleaner. Another good example are lens and traversals, they are generalizations of generic references such that CL can only envy.
Someone should really rewrite On Lisp and Paradigms of AI in Haskell.. (and also Design Patterns from GoF) I think it would prove this point.
You can even rerun portions of your code after you've modified it (rewind it), something that even Smalltalk never supported.
The JVM is a virtual machine targeted by many languages, some of which are dynamic, and Java isn't a "modern" static language from the standpoint of type theory and provability.
In Pharo Smalltalk, for example, you can pause running code in the debugger and modify it using syntax the language doesn't actually support, and then, in the same runtime, open a class browser and modify the compiler to support the new syntax before finally accepting your changes in the debugger and resuming execution. You can swap every reference to one object with references to another object using become:, effectively swapping the identities of two objects globally at runtime.
Perhaps all of this is possible in Haskell and the like, but it's still to be demonstrated.
> You can even rerun portions of your code after you've modified it (rewind it), something that even Smalltalk never supported.
Smalltalk and Lisp both support this.
Interesting, do you have a few pointers?
http://malisper.me/2015/07/07/debugging-lisp-part-1-recompil...
http://malisper.me/2015/07/14/debugging-lisp-part-2-inspecti...
http://malisper.me/2015/07/22/debugging-lisp-part-3-redefini...
http://malisper.me/2015/08/05/debugging-lisp-part-4-restarts...
http://malisper.me/2015/08/19/debugging-lisp-part-5-miscella...
I was specifically talking about debuggers that let you rerun code after you've modified it without having to restart your application, something that Java debuggers have allowed for almost ten years now.
I continue to think that neither Lisp nor Smalltalk offer this functionality, can someone prove this wrong?
Java got this from Lisp and Smalltalk. The first and third links covers what you're talking about. For Pharo, which is graphical, just open the image, open the debugger on some code and modify it to your liking before clicking "Proceed," or alternately, pick an earlier stack frame and do the same.
Did you even read this paragraph:
> In Pharo Smalltalk, for example, you can pause running code in the debugger and modify it using syntax the language doesn't actually support, and then, in the same runtime, open a class browser and modify the compiler to support the new syntax before finally accepting your changes in the debugger and resuming execution. You can swap every reference to one object with references to another object using become:, effectively swapping the identities of two objects globally at runtime.
Instead of making a case for "new successful dynamic languages" and why such new ones would emerge, both authors are defending the existing benefits of today's dynamic languages.
Elben Shira is saying that he predicts the next wave of successful languages will have static checking but also look & feel like a "dynamic" one made possible by smarter compiler technologies such as type inferencing. (Maxime Chevalier-Boisvert also mentions the new Crystal language as an example of this.)
If one is itemizing all the great points of Python/Ruby/Javascript, that's orthogonal to what Shira is betting on.
Erlang is reliable because it's fault tolerant. The Strong Static Typing camp wants software that is reliable because it is faultless.
Elixir has Erlang's philosophy on the problem. I think most of the wailing and gnashing of teeth is because most languages (even most statically typed ones historically) have just completely punted on the notion of language support for writing software that doesn't fall over in a stiff breeze. The more we build layers of software between humans and their actual goals the more this is going to come to a head.
Of course your compiler will never be able to proof the correctness (or lack of it) of all programs.
The relevant question is, what share of useful programs can our compilers help us creating? This is not answered yet, and you seem to have never even thought about it.
Yes, we will never be able to prove that a program is 100% correct but that doesn't mean we shouldn't prefer solutions that bring that number up (statically typed languages) over solutions that don't (dynamically typed languages).
Just because you can't do something perfectly doesn't mean you shouldn't try to do it as best you can.
In the real world, it may well be the case that while you're screwing around with a baroque type system, your competition has iterated three or four times and your milkshake has been drunk.
This is not a criticism of the language, simply descriptive of its current position. Go, which is substantially larger at the moment from what I can see, is just barely cracking that boundary that I described, and probably still has a good year or two before it's quite there.
Elixir is currently on a good trajectory, but I would definitely be concerned that trying to be a Ruby on top of the Erlang VM is going to make it a non-trivial challenge to get to the next level of usage, precisely because the Ruby space is covered, covered, and covered again with mature, existing languages. This is, again, not a promise from me it can't succeed, indeed I wish it all the best, just a description of its challenges. It is growing rapidly in its little niche but it may have a serious challenge getting to the next level.
And, back on topic, when this came up on Reddit, I'd observe that Elixir is the only dynamically-typed "up-and-coming" language I can think of. Nim is statically-typed. Rust is static. Go is probably on the way out of "up-and-coming" but it's static.
I also observe that all the dynamic languages are all adding static type support, even when the static type support is inevitably compromised by the dynamic portion of the language. In the meantime, I don't see any static language trying to move in a dynamic direction... in fact, many of the static languages are moving to be even more static! The net flow of feature work right now is strongly in the static direction.
C# added "dynamic" keyword in 4.0.
Also, I observe on the docs page this little note that says "As of Scala 2.10, defining direct or indirect subclasses of this trait is only possible if the language feature 'dynamics' is enabled."
Almost all the static languages have some sort of dynamic feature already, and generally have since day one. The mere existence of "dynamic" features isn't my point. The question is, how often are they used, and is the language tending towards more of them or less of them? I don't do Scala, so I can't speak to it, but does every little Scala program use this dynamic package, or is it something rarely reached for, and even perhaps considered a code smell by the community? If it's the latter, then it's not a huge notch in the "dynamic" belt.
I'm discussing the first derivative, not the current state of the world.
def lookup(name: String): Any
myObj.lookup("foo")
myObj.foo
But both of them are about type inference, not dynamic types.
People create languages all the time. Some languages (like PHP) were just created for problems that the creators had, not for some overriding empirical reason. Dynamic languages also tend to be easier to make. There's nothing to suggest people will stop creating languages and then adopting them, when they are tailored toward specific problems. Any reasoning as to why they might not appear is hand waving for attention.
The problem I outline is not that static typing involves additional boilerplate. It's the inherent additional cognitive load it introduces. A smarter compiler isn't going to help you there.
The fallacy of Elben's argument is in the assumption that there will be only advances in the field of static typing.
Meanwhile, people working with dynamic languages are improving their tools at an astonishing pace. For example, working with an editor like Cursive provides many refactoring options you could only expect from a static language a few years ago.
The biggest benefit though comes from interactivity though. Dynamic languages lend themselves extremely well to providing live coding environments. This allows for tools like the REPL and things like Figwheel https://github.com/bhauman/lein-figwheel and this kind workflow will be difficult to match for static languages.
Clojure and its family of language will be a thing of the past in a few years from now (Clojure is not even a thing of the present actually, given its minuscule mind share on the JVM).
Therefore, my prediction is that dynamic typing isn't going anywhere unless some empirical evidence appears that clearly demonstrates the advantages of static typing. Seeing how long static typing has been around I wouldn't hold my breath on that.
If you think that static languages have won then you must live in a very special bubble seeing how pretty much all of the web runs on JavaScript.
Having such strong convictions in absence of any empirical evidence is simply faith and static typing seems to just be a religious experience for some people. Many advocates are like bible thumpers coming to your door and telling you to accept monads into your heart. Naturally these are the loudest elements of the community who live in their own little echo chamber.
You seem to be missing that jasod's claim was that the "main claim of the article" is "that he predicts the next wave of successful languages will have static checking but also look & feel like a "dynamic" one made possible by smarter compiler technologies such as type inferencing." You haven't said anything in this thread about "the next wave of successful languages" one way or the other.
> If you think that static languages have won then you must live in a very special bubble seeing how pretty much all of the web runs on JavaScript.
That's true, but has nothing to do with the claim about "the next wave of successful languages" because javascript is clearly part of the last wave (or really, the one before).
You seem to be arguing (in an increasingly heated tone) a broader point (which may be right or wrong) in a thread attempting to focus on a more narrow one.
I think this is the counter-argument to the original claim; new dynamically typed languages can be successful by offering convenience for important functionality. Even so, I think type annotations to aid static analysis (which Elixir has and JS will likely have in the future) will be an important feature.
I think the bar has simply been raised for new languages with both static and dynamic type systems. The former can no longer be boilerplate-y, inflexible, and inconvenient, and the latter can no longer be statically uncertain and hard to reason about. This is a good thing.
A language like Haskell pretty much precludes having REPL integration as you can't just run any top level expression without going through the main.
What? Why not? I know people who pretty much use the REPL (GHCi) as their primary method of developing in Haskell.
Let's say I'm writing an endpoint to serve some data to the client. I would go to my db namespace, write the function to query the db, run the code and see what it's doing. I'll modify and run it until it does what I want. Then I might go write a controller and do the same thing, and so on.
At each point I have my code running within the context of the app, anything I write in the editor can be evaluated modified, and inspected.
This workflow simply doesn't exist in Haskell last I checked.
I've probably run most of the modules I've written in Haskell on the REPL, normally without importing the main one, because I use it for early-stage testing. I've run DB interfaces, network abstractions, and, of course, lots and lots of pure stuff. Just like you describe doing for Clojure.
Ok, I don't use an IDE that run stuff automatically at the REPL. On Haskell I import the interesting module, and run it. There is at least one IDE that connects with ghci, but I don't like it, and don't know how far it takes debuging (at a minimum, it evaluates your code and tells you the type of anything - I've tried this).
I simply have no idea how anyone would use a REPL to do main development (I never liked the idea anyway). I use it just for early stage testing and debugging assistance, just like I do in Python, Lisp, Prolog... and certainly not Perl - this one has a useless REPL.
With so many libraries supporting my development, I usually tend to write much less code using a dynamic language. Also most of the code tends to be isolated with clear interfaces exposed to other parts of the code. Writing tests is simpler too, compared to writing them in dynamic languages.
May be large projects with hundreds of thousands of lines of code, could benefit from statically typed languages. But for majority of the projects (80-20 rule) dynamic languages offer cleaner syntax, simpler coding experience and faster time to market (my opinion).
less confusion, less classes, more libraries, cleaner interfaces and easier testing.
Of those classes and libraries obviously apply per language not per typing discipline, confusion was rhetorically tied to the number of classes so again, thats about the language not the typing.
tormeh's right, the comparison was between languages, not between typing.
Solely for syntax, my assertion is that this is just because dynamic languages evolve faster. Because they are easier to build.
E.g. one person can buy the dragon book(s), use parser/lexer libraries, get an AST, and then start an interpreter than just stores objects as maps, and evals things dynamically.
But very few people are going to write a new/novel type system on their weekends.
So that's why, IMO, all the sexy/modern syntax shows up in dynamic languages first, and then takes ~a generation or two to reach static languages, where you need a dedicated team to put the effort into the type system + compiler + expected tool set (IDEs/etc.).
(E.g. see Scala, Kotlin, Rust, etc., which have all, IMO, achieved the "syntax as nice as Ruby" level, but took large teams of people to do it).
No, but one can hack an extension to the GHC type system in a weekend. (That's one of the reasons why there are so many.)
I've only reasonable experience with Rust in that list but... no. I've seen enough Scala to get that that's not true, and I know it's not true for Rust.
That's not to say that static languages can't have nice syntax - Crystal is a great example if you'r trying to match Ruby - but I don't think your examples are great.
With dynamic typing you're much more likely to break things up early precisely because you know you have to keep things in your head.
On the other hand, static typing allows you to just keep adding code and it makes sure that the code will compile and run. However, at some point you end up with a large and complex system that you don't really understand. All you know is that it's self consistent. Understanding what the business logic is doing and why can be quite difficult and the fact that it type checks does little to help you there.
Coincidentally this is the reason that static typing proponents are obsessed with refactoring. Once you realize what mess you got yourself in then you have a ton of code to refactor and there's no way you can keep track of it all in your head.
With a dynamic languages refactoring tends to happen in much shorter cycles and introduces much less cognitive load.
If you are programming with a language that forces to keep everything in your head, you'll never build big systems.
Now, I could just write the logic down as logical statements on paper, and check my code against it... Or I can write the logic down as logical statements on the computer, and have it check my code automatically. Pick your poison.
Having no type annotations, even in highly dynamic languages, leads to lower maintainability. However, some type systems are too complicated, and over-engineered, the best example of which is Haskell (compare it to Agda or Idris, which can do more, but have much much simpler type system).
It's not over-engineered. Making syntax in this way was a design goal. AFAIU, authors of the language thought that when you specify as much as possible in syntax, the code is easier to maintain. That's why it's so popular in enterprise world where code often lives for 10s of years.
That's not true of all cases. Most importantly, the number of abstractions that are baked into the language are a combination of workarounds (rather than fixing the type system) and whatever the JLS contributor's motivations are. Some of which has been useful, some of which has not. It's overengineered.
I suppose no rationale person believes dynamic language would disappear completely. Also I think both dynamic and statically typed languages exist on a spectrum. It is not very productive to talk in generalized terms like "end of dynamic languages".
I like dynamic languages, especially Python and Clojure. But programming with types is important to get things right, on the correct, formal basis. (I think functional programming is a misnomer, it should really be called algebraic-oriented programming, but I digress.) Once you see how to do it correctly it's perfectly OK to mimic the same abstraction in dynamic language.
Language like Haskell forces you to do it. It is more effort, but this effort pays off in getting composability and edge cases right. The end result is not actually harder to understand, it's rather that you did not fully understood the problem in the beginning (by glossing over technical details).
Static typing itself - no. What is important about Haskell is the equational reasoning (algebraic thinking, if you will) you can do, not static typing. Static typing is just a vehicle to be able to pull it off.
There is a payoff already - see my comment to Lisp guy above. We could also talk about how monads made LINQ great, or how streams in Java have to be more complicated than necessary because the language wasn't very well formally defined.
And there will be bigger payoff in the future, when we will learn for example how to type a whole big software component correctly (like a web service). So far, it's an art driven by engineering intuition; but formalization will help to make more robust systems.
The payoff is also assumed based on experience with other branches of mathematics (as I was alluding in my first comment), where historically, the formalization always paid off. It doesn't mean you cannot get things right informally, but it's usually fiendishly difficult.
Recently there was an article about a whole application: http://abailly.github.io/posts/cm-arch-design.html
I can also recommend book Functional Programming in Scala, which talks about how FP helps develop good API in more detail.
The analogy is more "Should I do Physics without computer-verified usage of units?", in which case the answer is "It depends."
For example, when I find it hard to understand Haskell code, it's not because of the type system, it's usually that the compiler is doing too much magic for me (with type classes, and automatically lifting things into monads).
I find that having the types actually makes the code itself easier to understand, because I can ask the compiler (usually through the REPL) to tell me what kind of thing can go where. It makes that kind of information more local.
I wonder what Shira counts as "successful"? Popular?
Sigh.