Clojure-powered Startups | Hacker News Reader

Clojure-powered Startups

infoq.com

139 pointspuredanger59 comments

Frankly, the selling point that clojure minimizes risk is not a compelling one. It sounds sexy, but its nonsensical.

Risk encompasses many aspects of technical decisions and there's a complicated set of interactions between these aspects. Saying a single language minimizes risk makes little sense.

For instance, if you're building a straight-up CRUD web app Ruby on Rails minimizes risk best. If you're building a realtime chat service node.js minimizes risk best (and I say this as a longtime detractor of node, its not the best thing engineering wise, but it's an 80% solution).

While I'm glad the speaker enjoyed success using Clojure, I don't think we need a slogan to promote Clojure, we need more killer platforms. We need problem-spaces where when someone asks "What's the highest quality implementation of X?" people say, well, clojure has XYZ, it's way ahead of the pack.

To my mind where clojure hits the sweet spot is in mixing performance with elegance. It's got a fantastic balance there for web apps. I would love for clojure to be the answer to the question of "our app servers are too slow". However, the ecosystem is just not there yet.

Between noir, immutant, aleph, and vert.x I think we have a bright future there, but whether clojure will hit critical mass to become a dominant platform anywhere is nigh impossible to predict.

      if you're building a straight-up CRUD web app Ruby
      on Rails minimizes risk best. If you're building a 
      realtime chat service node.js minimizes risk best

I do not agree with you on this one. I'm not into Node.js precisely because (for now) I consider it to increase the risk of any project.

Yes, your realtime chat will be easy to do with Node.js, but most web applications also need to talk to a database, and also need to do some background processing, and also need to communicate with third-party services through many protocols, sometimes hip, sometimes obscure and long-forgotten. And your app will also probably need at some point a freakishly boring admin filled with freakishly boring reports.

And building a multi-platform, multi-lingual project does lead to resource drainage and duplicate effort. This is one reason why Node.js is popular in the first place, because presumably people can share logic between the client and the server ; but in the larger context of things with Node.js I feel like I'm digging myself into a corner, just like I did with PHP several years ago.

Personally I'm starting to like the JVM more and more. Want to do a really scalable chat app in several lines of code? There's a solution for that [+]. Want to talk with the most obscure database in existence? The JVM can do it. Want to have your binaries work in 10 years from now? The JVM can do it. Want to use your hardware to the maximum? The JVM is the second best choice after C/C++. Want to make your sysops happy? The monitoring/profiling/deployment tools available for the JVM have absolutely no match.

[+] https://github.com/Atmosphere/atmosphere

I'm probably too cynical, but I think a big reason for node's popularity these days is the same as it was for rails 5-6 years ago - greenfield projects.

If I'm doing node now, it's not maintaining some legacy crap from 5 years ago - it's straight up new development. I get to do what I want. I don't think that factor comes in to the equation enough when people are talking about efficiencies and productivity metrics and such.

I still do a fair amount of PHP - my productivity maintaining PHP apps written by other people 5-10 years ago is pretty different from PHP I write from scratch, getting to use modern tools/libraries/techniques.

Yes, atmosphere is pretty cool - I'm looking at adding it to a project this fall. grails install-plugin atmosphere ;)

I don't think we need a slogan to promote Clojure, we need more killer platforms. We need problem-spaces where when someone asks "What's the highest quality implementation of X?" people say, well, clojure has XYZ, it's way ahead of the pack.

Datomic (http://www.infoq.com/presentations/The-Design-of-Datomic) is going to be one of Clojure's killer apps -- it adds durability to Clojure's immutable, persistent data structures, and it rethinks datastore design, from the ground up, to take advantage of modern storage services.

Datomic is super cool design-wise. It'd be great to see some apps actually using it.

Here is a shameless plug:

http://www.vixu.com (my startup) is based on open source Clojure/ClojureScript CMS (and soon webshop) software. See https://github.com/fmw/vix for the code (planning some refactoring over the summer before the first official release). Also doing search related stuff in Clojure (see https://github.com/fmw/alida for demo code I wrote for my EuroClojure presentation).

Clojure brought the fun back to software development for me and made me a better developer in general. It is also a very pragmatic choice, with the expressiveness of Lisp and plenty of reusable Java code being available.

There are two ways (and of course, the paths in between) you can run your startup

1 - You can go for the most popular languages: VB.NET, PHP, etc. Very easy to find people, lots of plugins, lots of support. This may be the way to go if you want something simple done in a small budget

2 - You can go for the niche languages: Haskell, Clojure, Go. that's more or less the "hipster hacker" way of doing things. Very difficult to find someone that can do even a Hello World, but once you find, they can write a scalable website in about 3 lines and connect to the newest nosql db in 2 lines more. This is good if you're the next twitter, fb, etc and your servers begin to melt even if serving login

(but then again, fb is in PHP and they added 'magic' in HipHop and infrastructure)

Yup Facebook and Yahoo are PHP. Google/Amazon/eBay/Walmart.com are all primarily a combination of Java and C++.

Although Facebook uses Erlang for chat.

... and Haskell for programmatically manipulating said PHP code base. [http://www.haskell.org/haskellwiki/Haskell_in_industry]

Facebook uses a combination of primarily PHP(1) (mostly front-end, rendering, some async jobs and scripts) and C++ (mostly back-end services, other bits of high-performance infrastructure, HipHop for PHP), with some Java (especially in the Hadoop, Hive, and similar areas), Python (mostly automation, scripts, infrastructure), and so forth.

From what I understand, Yahoo! has a similar mix of technologies.

In both cases, you really can't say "are PHP".

(1) Modulo some in-house and open source extensions (like XHP) running on HipHop for PHP.

I've hijacked a couple of components at Game Closure in Clojure, much to the chagrin of our CTO. (Just kidding, kinda.) For the places we've used it, Clojure has worked wonderfully. The only disadvantage is that few engineers know it, and few are willing to pick it up because they're afraid of it.

But if I were starting a new startup, I'd pick Clojure over anything else, hands-down.

The only disadvantage is that few engineers know it, and few are willing to pick it up because they're afraid of it.

That's the advantage.

Large junk of Polish startup Fablo (fablo.pl, e-commerce instant search engine based on natural language processing) is written in Clojure [1].

[1] http://jan.rychter.com/blog/2010/7/5/clojure-w-fablo.html

(nasty) typo: junk = chunk

Fablo is actually one of the most interesting startups on the Polish scene regarding technological challenges involved.

Hope I'm not hijacking this post, but does is the CLR version of Clojure any good? We have a lot of code in C#/F#/IronPython and am interested in trying some new components in Clojure just for kicks.

Is the CLR version have parity with the JVM version? Is it worth using at all?

Does it worth to write new components using Clojure if you already invested in C#, F#, IronPython?

Probably not. It's more of an exercise in learning Clojure - a language I've heard lots of good things about.

I find that I really get a feel for a language only if I'm building something in it that's actually useful (as opposed to just following along with Hello World and Factorial primers).

The problem with ClojureCLR is not the platform (that's 100% the same as Clojure on JVM) it's the tooling and lack of libs.

So instead of using DataOutputStream, you have to use BinaryOutputStream.

Clojure doesn't wrap much, so that means very little code written for one VM runs on the other. Plus you don't have lein, which is a bit problem tooling-wise

I don't think it has parity, for all of the reasons halgari noted, but I really don't feel that it offers much over F#, either. F# is a very solid answer to Clojure in all ways except macros: it solves similar problems in the same problem space, doing some things better and some things worse. I happen to have a slight preference for F#, but at any rate, I don't think you're missing anything by being on one or the other.

You are selling an ML variant for someone looking for a lisp variant. Good luck with that!

"Any sufficiently well-documented Lisp program contains an ML program in its comments."

(My Google-Fu is failing me; I can't find an attribution.)

I'm curious about how startups using Clojure are generating their HTML. Are you guys doing it in Clojure code (e.g. Noir) or are you using some sort of HTML templating support (e.g. JSP)?

While I really like the idea of writing HTML in Clojure, I like even more the idea of being able to have a HTML/CSS/etc expert, who doesn't necessarily know Clojure, be able to maintain the HTML instead of me.

Has anyone found what they think is a good solution?

> I like even more the idea of being able to have a HTML/CSS/etc expert, who doesn't necessarily know Clojure, be able to maintain the HTML instead of me.

Check out Enlive, it solves exactly this problem. HTML/CSS is separate from Clojure code and you fill out templates and stuff via CSS style selectors. It's sweet.

https://github.com/swannodette/enlive-tutorial

https://github.com/cgrand/enlive

Btw I think you can use Enlive with Noir, even though some of the examples from Noir use Hiccup, it's not a requirement.

That tutorial is truly excellent - I’ve been looking for something to replace Hiccup, and this looks to be it!

I prefer Enlive: https://github.com/cgrand/enlive

At Circle [1] are doing practically all of our HTML on the client side. Our server exposes an API using Noir, and we had to build a lot of our own stuff on top of that, like authentication. I added coffeescript and hamlcoffee support to dieter [2]. HTML emails are templated in mustache.

[1] https://circleci.com [2] https://github.com/edgecase/dieter

I'm a fan of mustache. There is an excellent implementation for Clojure: https://github.com/davidsantiago/stencil

I'm using hiccup for HTML.

Here I gave an overview of my experiment in building a web app entirely in Clojure: http://notehub.org/2012/6/16/how-notehub-is-built

Hotelicopter also runs on Clojure. And I would recommend Colin Steele's description of his decisions as CTO, as he took the codebase away from PHP, first to Ruby and then to Clojure. The bit about the failures of Ruby are (in my opinion) the most interesting. Basically, it was too much work to get Ruby to go fast enough, and their problems were solved by using Agents in Clojure.

This I think is the key bit of the story:

"At this point, we were still focused on using Ruby/Eventmachine as a cornerstone of our technology stack. But here we hit a snag. Yep. I’m gonna beat up Ruby, because it was a mistake.

...it doesn’t scale. ::ducks::

OK, that’s not completely fair. Ruby does scale. But it doesn’t scale well, or easily, and in benchmarking and stress testing, I was seeing that we were going to have a use a small truck load of resources at AWS, or spend a bunch of preciousssss developer time making it scale. It was too expensive. I wanted high user::machine density, and I didn’t want to have my developers do handstands to get it.

Yeah, I could’ve done lots of things. I didn’t have time to do that shit. Not with a four person team. Not with cash running out, and promises to keep, and no time to work for Ruby. I needed something that would work for me. This thing had to be FAST. It had to drive hardware to the limit without driving us crazy.

The bottom line is that it was too much work to make Ruby go fast enough.

...A little background on our application might help. Our job is to give prospective hotel bookers a view into what their options are. At the time we were making the decision to migrate from Ruby to Clojure, the system was using so-called “realtime” rates and availability checks with hotel suppliers to get that information. That meant that when a visitor conducted a search, we would spin up dozens of individual HTTP requests to hotel supplier sites to get rates and availability data at that very moment. We’d parse the responses, collate them, and present them back to the UI in just a few seconds.

You might ask why we didn’t cache that information, but suffice it to say there were significant business drivers for that decision.

Managing this concurrent (and long-running) IO was a major theme for us, and Ruby did so reasonably well using EventMachine. However, we had to normalize the returned data into a single unified data model, and none of our partners had simple (or compact) XML representations of the data, so not only did we have IO issues, but CPU-bound processing issues as well. The combination of the two made the EventMachine implementation suffer from less-then-stellar throughput, and because of Ruby’s green threads implementation and global interpreter lock, we had to run oodles of Ruby processes on each box to achieve reasonable throughput.

Perhaps just as importantly, the reactor pattern’s upside-down flow of control style of programming was (and is) a pain in the ass. It was hard to read, hard to maintain, and generally obstreperous.

..I can already year you Ruby folks protesting “Fibers!” and so on. Heh. Have fun storming that castle.

Clojure was a whole different story, and addressed these issues admirably, for all the reasons you’ll discover when you look into it further. "

http://www.colinsteele.org/post/23103789647/against-the-grai...

From this description it seems that Erlang/OTP would be even better choice than Clojure: we literally pack million of concurrect users into 1 high memory AWS instance.

Though Erlang not as modern or elegant as Clojure, but there is also LFE and Joxa.

It depends on how many external libraries you need. We considered both Clojure and Erlang in our project, ended up going with Clojure mostly because of available and easily accessible Java libraries.

I still find it amazing that I can:

1) add a single line to my project.clj (which will set up dependencies and download libs),

2) add a single line to my :import section at the top of the source file,

3) there is no point 3. Just call Java libraries.

But the choice really depends on your application.

Leiningen probably has its root from Maven, which IMHO the killer tool for Java. I've done .NET, a bit of Python, and RoR development/toying/experimentation in the past, and you need Bundler+Rake+RubyGem (at least) to emulate the core (important) feature of Maven.

Noticing that languages built on the Erlang VM seem to be more along the lines of "an X-like on the Erlang VM" instead of "X on the Erlang VM" (like JRuby, IronPython, ClojureScript). Is Erlang's VM less adaptable to existing languages vs. the JVM, etc.?

That's part of it, but it's also just that there's little motivation for using it as a general-purpose VM. Building a precise clone of Python on the Erlang VM would be more difficult than on JVM, but more importantly, it would be pointless — Python does not have the kind of deep-rooted concurrency semantics that are Erlang's raison d'etre. If you try to stick an existing language that doesn't closely resemble Erlang on the Erlang VM, all you're doing is creating a really awkward implementation of that language and not really gaining anything. So you end up with languages that are similar to X but with a huge transfusion of Erlang's semantics so that they're useful.

I don't know much about Erlang, can you expand on why you think it'd be a better fit? Is it just memory usage?

This is not new.

The default implementations of scripting languages do not scale, unless we making use of the advanced VMs or JIT techniques of alternative implementations.

A startup I was part of, made this discovery already back in 2002, when we moved our scripting based web framework, which was very much like Rails, but based on Apache/TCL, to the then early .NET beta.

Am I the only one who finds the InfoQ website too cluttered. I'd much rather watch this on Youtube.

Does it make sense to try using both Clojure and Rails in web app, Rails for UI and Clojure for more sophisticated business logic?

It should interact pretty well with JRuby on Rails, but I'm not sure you would get any benefit from using rails over a Clojure web framework like Noir.

I had a look at it but it's pretty young. I would prefer more mature framework.

Look at torquebox.org and immutant.org. You can run ruby and clojure on the same JVM and they can interop in a few ways.

It's possible to get quit a lot accomplished by loosely coupling Ring+Compojure, Enlive/Hiccup, and datastore modules like Clutch, Korma, clj-orient, etc.

I began my journey into Clojure about a month ago (after poking at it since last Sept) by diving into the "Clojure Programming" book recently published by O'Reilly. After finishing the book, I reached for Noir to fill my web-dev needs, as the idea of a "batteries included" framework appealed to me. But I ended up dropping Noir and adopting the approach explained in the book, i.e. loosely coupling the modules mentioned above.

Noir itself is built from some of those same modules, and it seems like a fine open source project, but I've found that I'm better off connecting the "lego blocks" together manually -- it's not a complex exercise and gives you a lot of freedom to experiment, learn and fine tune.

I've used Rails and Noir and they're not really comparable. Noir is much more minimalist and doesn't cover nearly the same ground as Rails. Noir is more like Sinatra as I understand it.

The Clojure mentality so far has eschewed all encompassing frameworks like Rails in favor of smaller interfitting libraries anyway.

I honestly think you should avoid this. We did this, and it's a nightmare, and we had to get rid of it.

Rails works very well if you stick to defaults. Each time you veer a little bit further from the defaults, things start to break (or at least require a lot more advanced knowledge to keep working).

We had a front-end in in JRuby-on-Rails, backend in clojure, in the same app/process. We had a "middle-end" which translated clojure->ruby and ruby->clojure, allowing us principally to use futures for parallelism and queuing. It worked really well at the start, but had tons of bugs. A nasty example was that we did email in SimpleMailer in Rails, but it was triggered from some backend functions, which in turn were triggered from the front-end. That meant that Ruby objects were being used in two different Ruby runtimes! Bugalicious.

Here's the library we extracted, if you really want to use this: https://github.com/circleci/cljr. My cofounder Allen talked about the experience at ClojureConj/west: https://github.com/arohner/clojurewest2012-slides/blob/maste...

Thanks for the correction, good to know. In the example you describe, the main mismatch seems to have been getting the two Ruby runtimes to communicate effectively. Was that a common theme?

I think the most important thing is "complexity is multiplicative". Every slight deviation from normal led to subtle errors that took days to fix, and kept coming back in different ways. The runtimes was just the most WTF way this happened.

There is actually another talk from Clojure/West on this topic that will be posted later on...

I would say yes if you build the Clojure component as a separate service that the Rails app makes calls to. Or I guess you could simply build it as a Java class and use Rails to execute it.

Yeah I am interested in the scenario with using Clojure-written web services.

Definitely. See my comment above about why not to do it in a single process.

this is what twitter is doing with scala i believe.

This is actually a fairly common choice for startups, the "deployment mullet". Business up front, party on the back end.

I believe the data is deceptive. Growth is fine and dandy but if you look at this graph http://www.indeed.com/jobtrends?q=clojure%2C+scala%2C+java&#... Tells a wholly different story than this one: http://www.indeed.com/jobtrends?q=clojure%2C+scala%2C+java&#...

Also didn't reddit switch from clojure/lisp-like language to python?

They switched from common lisp to python, which is an entirely different beast. CLisp is an awesome language, and obviously has the same roots in McCarthy's work, but they have different advantages/disadvantages (the library situation is better on the JVM, for example).

That was embarassing, and 20 minutes of my life I regret losing. Even the speaker appeared to be embarassed at times.

It's interesting that this thread has stayed on the front page of HN all day. I wonder how many people upvoted/participated in it without watching it? (I must admit, I participated without watching the whole thing.)

Noticed how your comment doesn't mention any actual complaints with the presentation?

It was vacuous, basically an extended resume for the presenter, mixed with empty platitudes about principles for clojure evangelism. And worst of all in terms of wasting my time, it was misrepresented. It comes across in the blurb as a talk about the contexts in which Clojure has been useful to startups, and the presenter represents it this way in the introduction, when instead it's "here are these vague descriptions of how I effectively evangelised Clojure in each of these startup companies."

Reply on news.ycombinator.com