Crowdfunding for Typed Clojure and ClojureScript | Hacker News Reader

Crowdfunding for Typed Clojure and ClojureScript

indiegogo.com

182 pointsswannodette36 comments

Ambrose is doing some super awesome work with Typed Clojure.

I'm a big believer in pluggable, optional type systems. Hell, even recovering typeaholics are starting to believe that Gilad Is Right [1]. Ambrose has managed to get me excited about what types can do for me in dynamic languages without bogging me down.

There's lots of cool things we can do with types without having them split our language in to two sublanguages (type and term languages), pervading our compilers and runtime environments, and preventing us from doing useful things. If you view types as yet one more tool in your verification toolbox, you gain a lot of freedom. Consider "Type Coverage" as an analogy to "Test Coverage". I shared that idea with Ambrose, and he jumped right on it! [2]

With good, community standardized tooling, we can forgo forcing the type system in to our compilers and down the throats of ever programmer, but still reserve the option when we need more confidence in our code. So get behind types as a library and throw a couple bucks at Ambrose so he can continue his work!

[1]: http://lambda-the-ultimate.org/node/1311

[2]: https://groups.google.com/d/topic/clojure/qiE7PP9uGKg/discus...

Making the type system pluggable and a separate pass also means you can do cool things like interfacing your type system with your database [1]. And the Heterogeneous and Value types mean you can be very flexible and precise with your requirements while also coding in a mostly idiomatic Clojure way. (Though things are obviously still raw or unimplemented, the base is looking good and shaping up quickly.)

[1]: https://gist.github.com/c-spencer/6569571 (just a PoC)

Yes, although that's not the only wayh to do these things: F# has type providers, for instance. Although I've been burnt by C# web services and Hibernate often enough to be sceptical of the idea.

Worth pointing out:

Type providers, while awesome, are simply code generators that run interleaved with the type checking pass.

What's happening in that linked Gist is a bit different: It's generating types signatures that are devoid of their own behavior. In order to recreate type providers (or Hibernate-like things), you'd also need to give a type as a parameter to a macro for code generation. Since you can invoke the type checker at anytime, you can interleave code generation and type checking in much the same way as F#.

I don't think it's a good idea to generate code and strongly coupled types at the same time. An alternative approach would be to take some common source data structure, generate code and then generate types to validate that code. Those are two separate transformations, not one like with type providers. Type providers mean that type checking, compilation, and execution are coupled. In this Gist, you could run the code with or without type checking and you could ignore the code generation completely and run type checking for internal consistency. Much more flexible.

I'm not a Clojure guy, but I'd love to see this type of funding for language/tool developers take off.

We as an industry/community get so much for free (and expect it these days), that it's ridiculous the only way for devs to get paid to do this is sell support contracts to enterprises (Redhat, Spring, TypeSafe). Or work for companies so embarrassingly profitable (Google) that they can subsidize it.

Neither of which is funds being provided to the developers themselves.

Not that I want to jump over to an MS-style "pay for everything" approach--open source is great, but seems like there could be a balance/middle ground that we're missing.

Completely agree - very excited to have just sponsored Ambrose's project as Snowplow Analytics. As an industry we need to get much better at acknowledging and supporting the role the technology components "under us" in the stack play in our own products' success; Zed Shaw made this exact point talking about Mongrel in his Why I (A/L)GPL post.[1]

So direct sponsorship to developers doing innovative things is great - and Typed Clojure is really innovative, one of the things I'm most excited about at the moment (along with Mesos, Rust and Summingbird). We don't use a lot of Clojure at Snowplow yet - but we use some[2], and we will likely use a lot more with Typed Clojure, because optional typing is such a great fit for ETL work.

Keep up the great work Ambrose!

[1] http://zedshaw.com/essays/why_i_gpl.html [2] https://github.com/snowplow/snowplow/tree/master/2-collector...

Backed.

Optional typing seems really promising, I'd like to hear more about what kinds of things it can't do though. If you hypothetically do annotate all your code, what are you missing out on by not having pervasive static typing built in like in Haskell? Is it mostly the fact that you're much less likely to go back and annotate all your code or are there deeper trade offs I'm not seeing since I'm not that experienced with strongly statically typed languages?

Does anyone know any good resources about the tradeoffs?

Yes, there are differences. Fundamentally, the Haskell type system is not just about correctness: it actually makes the language more expressive! You simply wouldn't be able to do some very awesome typeclass things with a system like this, for example. I've found things like QuickCheck--or even just using different monads--to be much more awkward without typeclasses.

The way this works is by allowing you to overload a function on its return type. It's very easy to imagine a function overloaded on its argument; for example, a hypothetical:

    to_string :: a -> String

This function can take an argument of some sort and returns its string representation. Many languages support this sort of overloading--it's very straightforward. But what about the symmetric from_string function?

    from_string :: String -> a

Now this is trickier! Most languages would force you to be more explicit about which function you're using, so you would have to write something like Double.from_string or Integer.from_string and so on. This loses the symmetry inherent in the types and forces you to add a whole bunch of redundant noise to your code. Very reminiscent of Java's Foo i = new Foo()!

With typeclasses, you can actually write a from_string function like this. Moreover, it works using exactly the same type inference system you already know and love! In practice, this means that the overloading is very straighforward: it just chooses whatever type you need. And since the type system is pervasive, you know that it will always choose the right instance if possible: if you use from_string expecting an int, it can't accidentally use Float.from_string because that wouldn't even typecheck.

This turns out to be useful in a whole bunch of different places. One of my favorites is with numeric literals: in Haskell, numeric literals are polymorphic. This means that if you write f 10, 10 will be the right type--integer, double, arbitrary precision integer, whatever--for the function. No more arbitrary sigils after your numbers (like 10L). More importantly, this makes new numeric types feel just like native ones[1]. So you can use normal literals with rational numbers or constructive real numbers or 8-bit words or symbolic bitvectors (most of which I've used in real code). These types no longer feel like second-class citizens!

Haskell type inference is even more useful than that though. Not only does it reduce the burden of using the type system, it also actually enables new workflows. Instead of giving the compiler your types and waiting for it to show you where they're wrong, you can actually just ask the compiler what type it expects. You can use this to interactively grow your program, informed by your types. The newest version of GHC will make this more convenient by adding a feature called "type holes", which lets you leave holes in your code and will tell you the types of those holes. Once this gets good editor support, it'll be incredible.

The Haskell type system--partly because it's pervasive and partly because it controls effects--also makes some high-level optimizations like steam fusion possible. The basic idea is that it can rewrite multiple logical traversals over an array into a single tight loop. The crazy thing is that this optimization is actually specified by a library, and not built into the compiler! So there are actually a whole bunch of rewrite optimizations like this, most of which are only guaranteed to be correct thanks to the type system. Since they're somewhat tricky to debug even with all the existing guarantees, I don't even want to imagine what it would be like with a weaker type system.

Honestly, I think you would be missing out on most of the real advantages Haskell gives you. And for what? At least in my experience, you do not get much of a boost in productivity or expressiveness. Even without invoking the optional type system.

So yeah: lots of deeper tradeoffs.

[1]: Take a look at "Growing a Language", a brilliant talk by Guy Steele that goes into why it's important to allow new additions to feel just like native parts of a language.

http://www.youtube.com/watch?v=_ahvzDzKdB0

All true, but at least Clojure (I don't know about other dynamic languages) provides some this power with protocols (which are very efficient) and multi-methods. core.logic provides the missing pieces.

> Honestly, I think you would be missing out on most of the real advantages Haskell gives you. And for what?

Forget about a static type system in particular: every bit of power you get from a language has a price. Sometimes the cost is performance and sometimes – and this is often a much higher cost – language complexity.

When choosing a programming language, you should not aim for the most powerful language, but for the one that gives you the power you need for a price you're willing to pay. Clojure strikes a pretty remarkable balance: a lot of power at a very low price (it's very simple, very easy to learn, integrates extremely well with legacy code, and can give you great performance when you need it). It is arguable whether Haskell gives you more power, but even assuming it does, this power comes at a much higher cost. Haskell is not easy to learn, and it's not simple to integrate with legacy code. The right question, I think, is how much are you willing to pay for extra power? Is extra power worth it?

Haskell, fundamentally, is actually quite simple: it's just the lambda calculus. You could fit the typing rules on a single piece of paper, and the evaluation rules are even more trivial. It is very difficult to come up with a simpler model than the lambda calculus.

Sure, it might be difficult to learn, but that's very different from being complex. Haskell is, if anything, different--but this is also why it's so much more expressive! I think this is a great example of Rich Hickey's distinction between simple and easy. Also, learning a language to some level of proficiency is an O(1) cost but the benefit is O(n) to how much you use it, so I really don't think you should consider that much.

Avoiding a language because it seems difficult to learn is just shortsighted.

Now, Haskell does have some complexity that other languages don't. But this is mostly in the compiler and runtime system: things like the incredible parallelism and concurrency features. There are languages that have much simpler runtimes, but Clojure ain't it: the JVM is probably the most complex existing runtime, full stop.

Legacy code, also, is more a matter of being on the JVM than language design. If you have a bunch of legacy code in C, it'll fit better with Haskell than Clojure. Calling out to native programs from the JVM is a real pain. The Haskell FFI, on the other hand, is relatively nice.

More generally, I think the idea that every bit of power has a price is not accurate. I agree that language design is going to be a complex optimization problem against several axes. However, this does not mean every improvement along one carries a cost along another--not every language is along the Pareto frontier! Since all languages are imperfect, you can get strict wins.

Also, as I understand it, protocols do the easy part I talked about. Certainly useful, but not very interesting. I do not see how you would accomplish return type polymorphism in Clojure--something like the from_string function or polymorphic numeric literals--because the information needed is simply not there. I'm not saying it's impossible, but I've certainly never seen it done.

Thanks.

There are certainly important tradeoffs. Haskell the language and Haskell the type system are designed to prevent many kinds of programming mistakes.

Typed Clojure is designed to follow any decisions Clojure makes. For example, it's not a type error to lookup a nonexistent key in a map, so we can't really catch typos in keys. Clojure allows dynamic code loading, so it would be rather fruitless to try completeness checking (and languages like Haskell can do it so much better).

You might want to watch my 2012 Clojure Conj talk. I think it touches briefly on this issue. I've always tried to be upfront about the limitations of Typed Clojure.

I apologise if that was a bit rambly, it's 5AM :)

> kinds of things it can't do

The most obvious thing an optional type system can't do is perform implicit searches that affect the semantics of the resulting program. Many of us consider this to be a good thing!

> what are you missing out on by not having pervasive static typing built in like in Haskell

One example of an implicit search is Haskell's return type polymorphism, where overloading is resolved by solving a logic puzzle to find a function with an unambiguous type signature match.

You can get around this problem statically or dynamically:

Statically #1: Explicit Search

Although probably overkill for something like resolving "read" to "readInteger" vs "readFloat", you can invoke a logic engine on a piece of code. This means that your language needs to support staging constructs, so that you can run code at various "times". Macros give us this, but there are potentially better mechanisms. You'll also need first class representations of previous loaded code, code under observation, the type solver, etc. People do things like this with core.logic in Clojure now: They generate a data structure, solve something about it, then either generate code from the solution or interpret some annotated data structure.

Statically #2: Explicit Parameterization

Just be explicit and write readFloat instead of read! If you need your code to be generic, you can use macros etc to fill in that slot at compile time.

Dynamically: OOP Techniques

See "Existential Types". As long as your coding to interfaces instead of concrete types, you can create proxy objects who can perform dynamic dispatch. What is a search problem at compile time is often a key lookup at runtime. This is what happens behind the scenes with generic monad code in GHC and friends anyway: They reify a dictionary of methods and thread that through.

It depends on the particular type system. For example in Dylan or Common Lisp (or Dart), annotating all types doesn't guarantee anything about whether you can still have type errors at runtime. In typed-clojure or typed-racket, you have a guarantee that if a type error does happen, it will happen either in dynamically-typed code, or on the boundary between dynamically and statically typed code.

Here's IMO a pretty freaking cool snippet of the possibilities, generating a static type from your database - http://gist.github.com/c-spencer/6569571

F# type providers already do that, also static types from CSV files, REST services and many others.

I didn't realize this until someone else mentioned it today. Thanks for bringing it up, very cool.

I explain how this differs from type providers here: https://news.ycombinator.com/item?id=6462374

Cool!

This is actually a really cool and decent crowd-funding campaign. It looks like it's going to get funded as well, I might even donate a few dollars myself.

Aside: Looks like Ambrose is an Australian guy as well. Spotted the accent immediately, as a fellow Australian myself it makes me proud to see another Australian who not only is super smart but has created something that I think is extremely useful. My experience with Typed Closure has been limited, but I think this deserves to be funded.

Thanks!

Is weird. If you have static types then want dynamic (I want it with Delphi). If have dynamic then want static (I want it with python).

I have thinking in how a language with both could be. For example, say Customer.Name(Str) is static but Customer..Name(?Str) is dynamic. In my experience, is better to have static typing almost all the time and "escape" to dynamic for auto-generate things (like data from a DB) and/or quick experiments...

isn't that what C# does with it's static dynamic type?

Is different because you need to type your untyped type!

I think in just a change in the "." or ".." character. So is not necessary to type the untyped, just change the dot for another thing to bypass the type checking in compile time

I hope it turns out better than optional typing for Erlang. I have frustrating memories of using it, which left me sceptic for optional typing features.

This is probably a stupid question, but I know that Ambrose has written and presented on a type checker written in core.logic. Is this related to that in any way?

Unrelated, Typed Clojure is based on research that went into Typed Racket

Just as cool though! :)

Is there a list of interesting practical use-cases for this that someone used to regular dynamic Clojure could look at to better understand the benefits?

It's invaluable with abstract code that throws around functions. It's also fantastic for documenting your code.

Watch "Typed Clojure" 2012 Conj talk, might reveal more about where it's best suited.

my first thought was getting 6 months of his time for 20K seems like an incredible bargain.

So why didn't Clojure have this from the beginning? Wasn't that on purpose?

Most of the research that Typed Clojure is based on didn't even exist when Clojure was created. Rich just made the best language he could, and waited for type theory to catch up, as it were!

See this goldmine of papers for Typed Racket http://www.ccs.neu.edu/home/samth/

Clojure is a small core of a language that can be flexibly extended with libraries in ways that other languages would require built-in support for. See core.async for another example of this.

Oh I see, I didn't know that. Thanks.

So why didn't humankind go to the stars from the very beginning? Wasn't that it's purpose?

As a major fan of Clojure and also of the possibilities of static typing, I have to give this campaign major props. I hope it funds.

Also, the perks are really cheap compared to the value, including exposure (because even though Clojure is a small community, it's a really strong community).

Reply on news.ycombinator.com