Comment by cbrozefsky | Hacker News Reader

Comment by cbrozefsky | Hacker News Reader

I've done two successful startups with lisp variants in my career, and continue to do clojure development at Cisco. The advantages of using lisps are real, if you stay focused on the problem domain and stay close to the customer -- you have great agility and flexibility. Woe to those who are tool jockeys and get too infatuated with their tools and code base, because there is a lot of rope to hang yourself and your teammates with.

The first was a Common Lisp web app for resource management, that was recently sold. Not a giant payout, but over it's entire life, starting in 2000, it paid peoples mortgages and put kids thru schools.

The second was a clojure based malware analysis engine, which we sold to Cisco about 3.5 years ago. That was a more conventional startup exit. We have scaled that into a global product. Additionally, we are developing even more clojure products within Cisco, and hiring...

I know from hiring clojure devs for the last 6 years that there are alot of companies, big and small, using it.

"Woe to those who are tool jockeys and get too infatuated with their tools and code base, because there is a lot of rope to hang yourself and your teammates with."

Could you expand a bit on this? I'm not entirely sure what you mean, but sense some good advice.

Not the grandparent poster, but stepping in.

Lisp, due to its easy metaprogramming, gives abstraction enthusiasts rather more rope to hang themselves than languages which make abstraction inconvenient (e.g. Go, which has a simple type system, or Java, which is very verbose). So, if one isn't careful, it's possible to write a very clean, very nice Lisp DSL which is totally incomprehensible to anybody but the author, and thus hell to maintain over time.

Relevant quote, from https://www.joelonsoftware.com/2000/07/22/microsoft-goes-bon... –

> When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there's a general pattern: sending files. That's one level of abstraction already. Then they go up one more level: people send files, but web browsers also "send" requests for web pages. Those are both sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it's getting really vague and nobody really knows what they're talking about any more.

> And if you go too far up, abstraction-wise, you run out of oxygen. Sometimes smart thinkers just don't know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don't actually mean anything at all.

More here: http://wiki.c2.com/?TooMuchAbstraction

>So, if one isn't careful, it's possible to write a very clean, very nice Lisp DSL which is totally incomprehensible to anybody but the author, and thus hell to maintain over time.

This is a myth. DSLs in Lisp are almost always done in Lisp syntax, no surprises. The idea of having a higher abstraction is to make the problem domain map more clearly to the code.

>totally incomprehensible to anybody but the author

Now, what people who like to repeat this myth forget to say, is that when using an OOP language, say, Java, and using another's person code, one also needs to learn the classes that this person has created, the methods and how they interact. And this can be "totally incomprehensible to anybody but the author".

>Relevant quote

No, not relevant. Joel Spolsky is talking about device abstractions.

>>And this can be "totally incomprehensible to anybody but the author".

It always is.

Code isn't magic, so you have to write whatever logic you would write in one language in another, for any application you would write.

The only exception if your language provides abstractions on which you can build on. For example in Perl, you can do a lot of file operations and regex work without libraries. In Java you might need a library, sometimes a combination of libraries, whose documentation one might have to read.

Same goes with lisp too. The only difference being, every once in a while one might discover a way to do large patterns of code through simple ways using macros.

professional clojure programmer here, not a myth, DSLs often abstract in the wrong direction and by departing from a language of values and function composition (eg by introducing macros) a bad dsl in fact becomes a barrier to good abstraction. Consider any DSL for html generation that predates React.js; react isn’t even a DSL at all, react is values and functions, so there’s all this good innovation happening around React that was all dead ends on 100% of pre-React DSLs.

>a bad dsl in fact becomes a barrier to good abstraction. Consider any DSL for html generation that predates React.js; react isn’t even a DSL at all, react is values and functions

React isn't a DSL; React includes a DSL for html generation.

And by the way, we have had such a thing in Lisp since the existence of HTML. Tons of libraries that allow you to inline HTML into your code.

>and by departing from a language of values and function composition (eg by introducing macros)

You are assuming that a DSL is done by using macros; which is incorrect; in Lisp you can easily create your own DSL using plain functions as well.

Sounds like maybe we're talking past each other. Would you mind clarifying how you're defining a DSL?

I'm not fully sure what you're disagreeing with. Of course it's possible to write an overly abstracted DSL in Lisp, as it also is in Java or Go.

> The idea of having a higher abstraction is to make the problem domain map more clearly to the code.

Certainly. And, as per the Spolsky quote, too much abstraction can actually miss the expressivity sweet spot and start to obscure the problem domain.

> No, not relevant.

It is if you, ah, abstract away the specific context of his story.

> So, if one isn't careful, it's possible to write a very clean, very nice Lisp DSL which is totally incomprehensible to anybody but the author, and thus hell to maintain over time.

You can't really call it a "very nice" DSL if it is incomprehensible and unmaintainable :)

What you have described is one issue, but it really doesn't have a lot to do with lisp DSLs. You can make that mistake in any language.

A problem that is more specific to lisps is that since it really is possible to write beautiful programs, there is a temptation to spend a lot of time designing DSLs or otherwise refactoring your program to make it very clear, concise and maintainable - but without actually making it more useful. You may spend hours (or weeks) improving your code without actually improving functionality at all.

This is less of a temptation in Java, for example, because it's such a rubbish language to program in in the first place there's very little temptation to spend time refining your code. You just accept that your program is never going to feel like a work of art no matter what you do, and get on with actually producing a useful solution.

I made a nice Lisp with a comprehensive macro system.

I also use it to handle my contracting finances. Let's grep that software for "macro":

  $ grep 'macro' *.tl
  money.tl:(defmacro define-relational (name binary)
  money.tl:(defmacro define-arith-predicate (name usr-package-fun)
  money.tl:(defmacro define-unary (name usr-package-fun)
  time.tl:(defmacro def-date-var (name fallback-val . date- range-val-triplets)
  time.tl:         (defsymacro ,name (range-lookup *date* ,fb ,rvt))))

Five hits; that's it! Three macros in money; they are all local shorthands just to eliminate some repetitive code. One macro in time for defining a date-specific variable, and that macro introduces a global symbol macro when invoked.

The bulk of the system is OOP code: accounts, transactions, expenses, invoices, deltas, ... things with boring stuff like slots and methods. (Of course, I wrote that object system with a fair bit of contribution from macros; but that's in the language, not in the project).

The DSL for recording transactions consists of plain old functions with transparent contents: just instantiate certain objects, do certain calculations and insert into the ledger.

The date-specific variable thing is cool; it lets us define a variable, such as an income tax deduction rate, which has different values based on what date it is. Not the current system date, but the context date for the creation of a transaction: the date to which it is being accrued. A macro helps define that with a clear syntax, and the symbol macro makes it look like an ordinary variable.

Too be fair, metaprogramming is just one additional rope. I've seen enough OOP-overabstraction in commercial CL code to waste a few lifetimes with maintenance. Worse, code written by people who understand AMOP and they made full use of it[1].

I think the advice given in this thread's root is good: Stay close to the problem domain. Really close. If you wonder whether you have gone too far, you probably have.

[1] You of course can gather HTTP-headers via :list method combination. Seriously, you can. And then comes reality in the form of some closed source middleware that expects headers in a specific order and it even refuses to discuss elegance or adhering to protocol specs with you.

Out of sheer curiosity, is Chez Scheme still used at Cisco?

You cannot use data as a program in Clojure, can you? I mean is data also a program in Clojure?

You can, and yes, it is.

Reply on news.ycombinator.com