Hacker News Reader

Comment by Townley | Hacker News Reader

I can, and do, appreciate opinions on languages themselves. But my decision to continue using Python over Clojure is much less about the language's syntax and more about the available libraries. How do Clojure web frameworks compare to Django/Flask? What would I use instead of Pandas for data analysis? Are there keras-like deep learning tools?

Switching languages based on a comparison of idiosyncrasies between the two seems like moving to a new city based solely on the weather: a valid consideration, but nowhere near sufficient.

View on HN

Clojure generally has pretty good library support: the JVM has several high-quality web application servers: Jetty, Netty, Tomcat, Glassfish, etc. And, being a Lisp, Clojure can wrap these with a nice interface. The same is true in other areas: you have very nice access to the entire JVM ecosystem and really-well designed interop with the host language's concepts so, to a large degree, you never run into a situation where you can't do something because there isn't a good library for it: at work, we regularly use Kafka, Avro, Elasticsearch and several other major data systems and we mostly just use thin wrappers around the official Java implementations of the clients for these systems.

Similarly, when connecting to a database like Oracle, it's trivial to just use JDBC/etc. while, for example, my coworkers who are using Haskell generally have to write a Java gateway because there aren't well-tested clients for Haskell.

Curious if your coworkers have looked Eta, a Haskell on the JVM?

Also curious how your Haskellers and Clojurers are peaceably coexisting? (I ask this lightly but still curious :)

I've talked to them a little bit about Eta and it sounds like there are some concerns about the stability/maturity of the JVM FFI interface, among other things.

I was unaware of this and have been playing with Elm and Haskell in my spare time! Neat to know that there's a Haskell like language on the JVM! Will have to check it out :)

I agree. Within the data science community I often see the question of "R versus Python" and it always makes me think of the Old El Paso commercial: ¿por qué no los dos?

I know choosing the right tools for the task can have serious long-term implications on the project, team, or organization level, but individually I get the impression that people worry too much about which language is best when, in reality, many of them are perfectly suitable and appropriate.

For us, the answer to "R vs Python" is "Spark" since eventually you get to dara sizes where just R and Python don't really work.

> where just R and Python don't really work.

From the Spark website: "Write applications quickly in Java, Scala, Python, R, and SQL".

Thats not really how it works. You can write spark code in all those languages, but it's still spark code, running on spark execution engine. When you do stuff, it's on spark dataframes, not on R or Python dataframes.

There's limited interaction with pandas but that's about it - you don't get R statistical algoritms for example.

(I just realised that my previous comment may have come out as rude, I'm sorry for that. I was typing in a hurry and it came out a bit dry).

Yes I agree with your points, what I was trying to convey in my comment is that, since Spark is not a programming language, I think that comparing it to Python or R is a bit of an apples to oranges comparison. The other point that I was trying to make is that both of those languages could be described as first-class citizens within the Spark ecosystem.

Clojure works on the JVM so you have native access to all of the Java ecosystem. With that in mind I'd say the libraries are actually better for Clojure with the exception of data analysis. I don't think there's a language out there that has as many data libraries as Python (like Pandas).

Aren't there a lot of data libraries for scala?

Yeah, at the very least, there's Spark, which is reasonably nice to use from Clojure via the Java API:

https://github.com/finalfantasia/apache-spark-examples-in-cl...

Yes but more data engineering than data science (someone else mentioned Spark).

I really don't understand this point about just using Java libraries in Clojure. Why not just use Java? Wouldn't the experience of writing Clojure that fits the Java library's model be horrible?

Not really. Clojure has nice native syntax for Java interop (or JavaScript for ClojureScript). In fact, I'll occasionally use Clojure to explore a Java API while working in Java.

What happens in my experience is that Clojure programs do tend to encapsulate more extensive use of Java interop into their own modules. For more standard or commonly used packages, this means a canned library or similar. (But this notion of abstracting out APIs is just good software engineering in general, and not really specific to Clojure or anything else.)

The main points being:

1. If you need it, and there's not a Clojure alternative, you don't have to write your own. 2. If you have any proprietary .jars or anything like that, you can leverage those services, models, etc when building an existing Clojure app. This is mainly a transition-type thing, in my experience. "Well, we could try Clojure but then we cant reuse any of our existing models." Not entirely true. However, you probably will want to change those from mutable classes pretty quickly :)

Yes, you are suddenly writing java with parentheses.

Don't get me wrong, Clojure is a great language in itself, but the lack of libs make you spent 90% of your time writing interop, you start to ask yourself if it's worth the trouble.

I actually find Clojure's `doto` notation to be the cleanest way to interact pure Java. It's like an improved syntax for Java that's native in Clojure.

When comparing programming languages / tech stacks, I am also more interested in the full picture (libraries, complexity, performance) than just differences in syntactic sugar. I use https://github.com/gengstrand/clojure-news-feed as a test bed for such evaluations. There, you will find almost feature identical microservices in clojure using ring with the jetty adaptor and python 3 on flask.

The clojure service was in 13 files with a total of 710 Lines of Code. The python service was in 33 files with a total of 1561 LoC.

I also have published load test results of these microservices at http://glennengstrand.info/software/performance/eks/gke where you will find the per minute average throughput for these two microservices to be about the same but the clojure version was about twice as slow as the python version.

I wouldn't say your Python example is comparable to the Clojure code. I'd describe it as "writing java in .py files", actually. No one is forcing you to write classes for everything in Python, having one class per file, and insisting to deserialize everything into classes. Simple functions and native data structures work just as well as in Clojure, and you'd end up with similar code as a result.

rails/Django -> luminous

Flask -> pedestal/clojupture

Relay -> fulcro

Pandas -> clojure.core, specter

Keras -> mxnext/neanderthal

The only one I miss in Python that is not in any other package is matplotlib. Is there any clojure alternatives?

I've only very briefly used it and am by no means an expert, but perhaps Incanter?[1] Specifically the Plotting section on the Data Sorcery page[2] has more information.

[1] http://incanter.org/

[2] https://data-sorcery.org/contents/

Incanter is a fantastic toolkit. A great suggestion!

there is a "work in progress" thing named "REBL" REBL is "Read Eval Browse Loop". It's a kind of interactive view of any data.

Basead on it's shape, you can extend and create many views.

I don't think luminous really compares to rails/django. Luminus is more like a great starting point, with some boilerplate and glue between libraries, while rails/django is way more batteries included.

Blog Post Author Here:

Personal Opinion: Django is a nightmare to work with. It's too opinionated and a pain in the ass. However, I love Flask. I wouldn't be where I am today had I not found Flask in 2012/2013.

If you're looking for web technologies in Clojure, you'll find quite a few good ones. However, I say web technologies instead of frameworks because generally, the Clojure community prefers to compose libraries to suit their needs and not leverage an opinionated framework. However, if you'd like someone to do that composing for you, the Luminus web template is fantastic.

http://www.luminusweb.net/

I'd argue that the JVM is a good move from python. There are a number of rich frame works on the JVM with great tooling. Also if you use PyCharm switching IDEs is easy with Intellij. I switched from python -> F# -> Kotlin.

I do agree the language is part of the battle. The other is tooling, and libraries. F#/dotnet was painful as a linux dev. The JVM works on almost any platform with great tooling.

For Libraries. I haven't found anything akin to django admin, but I don't really miss it. For me

SQLAlchemy -> Jooq

Flask -> Spark/Vertx

I like writing Kotlin, but I feel like I'm still struggling with the "functional" side it. I think I just like writing java too much.

> F#/dotnet was painful as a linux dev.

Which part made made it painful the most?

was F# painful pre-dotnet core? Have you tried it since? I ran into issues with F# and Mono a few years back and haven't tried since core came out.

I will admit that there aren't as many direct Clojure libraries, but many of the core things out there; core.matrix works pretty well for elaborate matrix processing, there are several decent web frameworks, as well as parsing libraries.

There's a metric ton of libraries for Java and Scala out there, and as a consequence of Clojure's ability to use Java stuff, you have access to all of it. I admit I've not done a ton of machine-learning stuff, but I've used Spark ML, which works well.

The data science story in Clojure is good and improving fast. Neanderthal is the fastest matrix library available on the JVM. Bayadera is ridiculously fast probabilistic programming (not to mention Anglican, which has a slightly easier API in exchange for perf). There's a lot of activity around Clojure bindings for MXNet, if you want deep learning. It's not mainstream Python, but it's far from limited by community or libraries.

Clojure lets you leverage the entire Java library ecosystem, which is flippin' huge -- bigger, even, than Python's.

Except for numerics, which is my whole world, where Java is horrific roadkill. FWIIW back 8 years ago I was ---><--- this close to trying to build out an important system with wrapped Fortran in Clojure. People (who even then looked like they didn't know what they were talking about) assured me the JVM would eventually surpass lapack, because MUH JVM, and that has failed to come to pass.

The SciPy stack got me into Python and it making it hard to leave despite becoming 100% fed up with Python's GIL. I need to port some signal processing code I prototyped in Python to Java for a customer, and a survey of libraries for Java is not making me look forward to this effort.

I'd say Clojure has just the right amount of syntactic friction to calling into java, that encourages people to write Clojurey libraries (or wrappers for Java libs) and keep the Javaisms at bay from your code.

Regarding what is available in the Java/JVM ecosystem, here are some Maven Central stats[0]:

  Total number of artifacts indexed (GAV): 3,566,566
  Total number of unique artifacts indexed (GA): 272,507

0 - https://search.maven.org/stats

Reply on news.ycombinator.com