Comment by kelseyfrog | Hacker News Reader

Comment by kelseyfrog | Hacker News Reader

Parent Full thread

Clojure user here since 2010(my earliest Clojure project on Github), and while I agree with the fun point, the iceberg wart for me at this point is the inelegance of the interface hierarchy and its structure behind the scenes.

Clojure's forward-facing interface (a hundred functions that operate on one data structure) ended up breaking down for me at some point and became 10 functions on 10 data structures and those data structures became AFn, APersistentSet, APersistentMap, APersistentVector, IFn, IPersistentSet, IPersistentMap, IPersistentVector, ITransientMap, ITransientVector, IndexedSeq, LazySeq, &c, &c, &c.

Maybe I started writing code wrong. Maybe I dived too deep into the internals, Maybe it was something else. But at the end, there wasn't one data structure, there were dozens and dozens each with justifiable differences, but even so, that's not what was advertised. It took a decade to reach that point and perhaps the vast majority of folks never will.

The sad part is that I know none of it is up for change without creating a Clojure2, and that highlights the problem. Why should changing the internals need to break backwards compatibility? There is one unspeakable reason: the illusion wasn't complete and they weren't really internals to begin with.

I've been programming with Clojure since 2010 and using it in production almost as long and I hardly ever care about the internal types so I'm really curious as to how you went down that "rabbit hole"? What sort of problems were you solving that necessitated delving into the implementation details behind the abstractions?

I've only needed to dig into that occasionally for a handful of specific situations (for example, in next.jdbc, where I create a hash map like abstraction over the (mutable) ResultSet object from Java -- to paper over some nasty interop issues).

What is the idea behind immutable variant of a ResultSet? Since it maintains a database cursor, does it make sense to have it immutable?

> A ResultSet object maintains a cursor pointing to its current row of data. Initially the cursor is positioned before the first row [0]

0 - https://docs.oracle.com/javase/7/docs/api/java/sql/ResultSet...

If you're a Clojure developer and you're using the next.jdbc library you don't need to ask that question :)

Obviously, I'm not :)

Lets say that someone wants to improve on Clojure and make a better functional, immutable LISP! What should they start with ? Some things I miss is type support (for easy refactoring, auto documentation and performance), (small) native compilation, support for mobile platforms and their UI's, first-class web-assembly support, real structs and misc things like performant implementations for `first`, `last` and other clojure functions, light-weight concurrency like Go, data-flow analysis and pipelines, etc.

An example of clojure performance issues covered here: https://tech.redplanetlabs.com/2020/09/02/clojure-faster/

All that stuff above should be fixed at the language/stdlib level.

But what is your take on what needs fixing ? What would have been your solution for the N data-structures/ M algo problem you mentioned above ?

If you want an easy win, one thing Rich mentioned in a talk is - if he had to do Clojure all over again today - he would put transducers at the 'bottom'. For data transformation this makes the underlying collection type largely immaterial.

> real structs

I'm not even sure if value types are conducive towards immutable, persistent data structures. I'm certainly excited for project Valhalla but I'm not sure if Clojure, nor any Clojure-like JVM language, written in an idiomatic fashion, would really benefit from it.

> light-weight concurrency like Go

Project loom is already in preview mode. Lightweight concurrency is nearly here for any JVM language, and once it's fully released I will likely have zero reason to use core.async.

> type support

Static typing is A Thing you can choose to do, but I doubt you would get many daily Clojure users agreeing that it is "better". I think it's different and better in some circumstances but not necessarily others. It definitely feels trendy these days, sorta like how dynamic typing felt trendy 15 years ago.

Yes on the type support not really attracting Clojure users. If you wanted to enable better refactoring, the thing to do would be to take more pages from other lisps and use the compiler to add more metadata to the runtime (in development mode at least) so that you can get better tracking of the language constructs.

Clojure users fighting against static typing are on the wrong side of history. They are fighting a side for all the wrong reasons, and they will lose.

There is a reason why all dynamically typed languages today are scrambling to add some form of static typing to their language, but never the other way around.

Static typing does everything dynamic typing does, but better, faster, allow automatic refactoring, faster programs, better navigation, better documentation, better maintenance.

I also believe the same thing.

However, to be fair, Clojure's response to types is `org.clojure/spec.alpha`, and if you give it a really good chance, you'll see that it brings a lot to the table.

It doesn't change that I believe strong typing (a Haskell or Idris/Agda-like flavour of it) to be the future, but what it did change was how I thought about type systems and how to write and test programs.

Or maybe it isn’t a war, there isn’t anyone fighting, and there’s nothing to be lost.

I understand the point you're making, and that you wanted to make it dramatically. I got a nice belly laugh thinking about my future grandchildren chastising me for forsaking static types:

"All your life's work for naught, Pepaw. If only you had been on the right side of history".

I do have a substantive disagreement with this statement: "There is a reason why all dynamically typed languages today are scrambling to add some form of static typing to their language, but never the other way around".

Languages like C#, Java, Scala, and Typescript have all adopted some degree of type inference. Their users wanted it for a long time. To begin with, none of these languages are anywhere near as strictly-typed as a language like Haskell. Clearly even developers on "the right side of history" don't want to maximize static type checking in all cases, so the issue is not as cut and dry as you make it sound.

You are wrong. https://www.youtube.com/watch?v=YR5WdGrpoug Static typing only leads to devs trying to press the real world in stupid arbitrary categories.

Yeah and spec isn’t trying to press the real world into arbitrary categories? /s

The thing is, having some structure in terms of types or data shape constraints helps you model and understand your program. You can change them as your understand of your program changes or the need arises. You can be against static types because they force you to slow down and think about things before you can just get to coding, and it can be sometimes annoying to model your system using them (and sometimes not so useful), or other technical reasons like some type systems don’t allow you to model certain things, but to say it just leads to devs putting the real world into stupid, arbitrary categories when the real world (and your program!) primarily have data that can be trivially categorized because that’s what Rich Hickey said makes you sound like cargo-culting or parroting a point without diving deeper into the specific reasons and potential counterarguments.

IMHO, external schema to validate structures at entry point is the way to go.

We have to have both: types and external schemas in oo languages.

Plus, types make working with maps cumbersome and force you to maybe unnecessary data cages called dto-s

> But what is your take on what needs fixing ?

I appreciate that it's a bit frustrating to have someone vaguely identify an area of concern and clarification and actionable suggestions are much more valuable.

There's two things. One is that the data structure ontology[1] is difficult to internalize because of its complexity. When I write Clojure code, I consider the capabilities of the data structures being passed to the function. I think we have some implicit understanding of this early on. You expect the type of such and such to support seq-ing or deref-ing or something else. You're thinking in terms of capabilities and that's good. But if you want to be exacting, at some point you'll have to be able to point to an interface or several in the diagram and say, "yes, I really do expect something that conforms to IPersistentVector or IMeta or something else. In order to preserve backwards compatibility, this is more confusing than it needs to be. Moreover, at that point, you're not in Clojure-land anymore.

My second point is that at this step in the narrative, you're reading Java code and the illusion of 100 functions is broken. I ask myself, "Why, aren't Clojure data structures implemented in terms of protocols?" There's a perfectly good way of abstracting functionality that seems good enough for users, but not good enough for standard library implementers? It feels like the answer is, "We don't want to break Clojure1.x and we don't want to deal with the Python3 problem." Which is fair, admirable even, but it still leaves me left wanting. I think there's a simpler, more consistent Clojure hiding in there, waiting to be let out.

Apologies for the awkward way this reads; I simply wanted to get it out before lunch time.

1. https://raw.githubusercontent.com/jafingerhut/clojure-classe...

I don't want this to sound hostile because I have a similar love/hate relationship to Clojure, I truly don't understand what you're arguing. Or, it seems like your issue is with lack of static types rather than an issue with the implementation? I've written a lot of Clojure and have an okay to pretty good understanding of the implementation, but it's just hardly ever relevant to the code I write, so I still don't totally understand what you're getting at exactly. Like, what's the actual problem here? Just that there's more Java "intrinsics" than pure Clojure code? I guess I do agree that Clojure is really just Java, and it would do better for people to think about it just as (really nice) syntax sugar over Java, but I guess I don't see that as a problem, but rather the absolute biggest strength of Clojrue.

I think he is arguing for more protocols than simply sequence. That is a sequence can be attributed fine-granular capabilities represented by protocols and that allows generic functions to be tailored towards those capabilities for improved performance.

Are you really suggesting type support? I don't mind types but I think spec is more in the spirit of Clojure. Have you taken a look at it? https://clojure.org/about/spec

spec is mostly useless for tools and IDE's and static analysis. Spec necessitates running your program with an arbitrary runtime complexity. Type checking is done before your program runs. The idea is to remove the need for runtime checks as much as possible.

But clojure is not meant to be developed statically! It is supposed to be running while developing it. You don't need static analysis if the programm is running and inspectable.

Ah yes thanks for the elaboration. That's what I meant by the spirit of Clojure.

Right ! But you still had fun the last decade correct /s ? :P

My "real response" to is, I haven't seen this complaint(concern) to this extend that you describe in the wild or in my own life. I've definitely not coded in Clojure long enough to have seen any of that like you have, thus far my experience has been good with 'just' using maps.

I do feel my next "step-up" would be to incorporate something like SPEC or Mali to "define/check" the fields/structure of said maps.

An interesting complaint. What problem domain(s) do you work in? It sounds like you've found something that Clojure isn't a great fit for.

The mix of interfaces would be a huge pain if you needed their specific behaviours, but in my experience they've never mattered and lings like LazySeq just act as performance optimisations.

Do you have some os projects out there to share where you had to do that? Makes me curious, I think the only time I've ended up making a specific data structure in clojure was for a library for perf reasons. In application code basically never.

Specifically it was using GraalVM to interop Clojure on one side with React running on GraalJS on the other. It definitely pushed the limits of what is or even ought to be possible, but at the same time highlighted those very issues. I'll be the first to admit that this is an extreme edge case, but that doesn't take away from the fact that the case is still there.

Reply on news.ycombinator.com