Hacker News Reader

Comment by speckledjim | Hacker News Reader

Either you care about shipping products, and making things of value, or you care about what the latest fashionable language to do it in is.

View on HN

That brings us to point (1) of his article. He asks us to make a large investment in learning a language that might pay off in the future... might.

My understanding of Scala is quite imperfect but the more I've learned the more it seems that it's got pretty facades built over weak wood. Rather than being a 'scalable language' it comes across like a fake town that was built as a set for a Western. Look at any beautiful concise code sample that they try to seduce you with and you find that any small change requires tripling the code size. Look at the chapter in the book on DSLs and the take-away is "don't try to build internal DSLs".

Be it looking at the profiler running on on a Scala program that uses actors or finding that I never quite seem to get away with not having to declare types explicitly, I have a facepalm event every hour or so working with Scala.

In my case, (2) has nothing to do with it. I'm quite familiar with functional programming, both in languages that support it well and languages that don't. These days even my PHP looks like LISP.

(3) is certainly true, and I think it has something to do with the cultural difference that @speckledjim points out.

Anyone who's got to work with Scala will definitely confront (4); if you're working in Java or PHP or some other commercially viable language you can probably solve a high fraction of problems by using Google and Stack Overflow.

If you're programming in Scala you'll have a lot more success understanding the fundamentals of the language. If, for instance, you need to access Scala objects from Java (particularly the super-prevalent ones like Option and List) you'll find that cookbook-style documentation is 100% AWOL. Go look at the scaladocs for Option and List, however, and you'll be calling methods on them in no time from Java.

If you insist on working with Scala, get a book and read it all the way through from the beginning, skipping forward when you don't understand something.

    Look at the chapter in the book on DSLs
    and the take-away is "don't try to build 
    internal DSLs".

Which book do you mean? Also, do you mind elaborating on why one should avoid building DSLs with Scala? In my own experience [1] I've found it extremely nice for internal DSLs.

[1]: https://github.com/fogus/baysick

Presumably he means Ordersky's Programming Scala. While it is possibly one of the better Scala books available it is downright bad, having committed the doorstop producing sin of confusing a language tutorial with a reference with a thesis defense. The second edition of the text is all over the place, providing little in the way of even trivial program examples and maintaining shockingly conservative advice from the first edition, written for the language in its state a few versions back.

The assessment that Scala is a pretty structure built of weak wood is not too far off, at this point. Compared to other strongly functional languages--Haskell and ML, say--the type system is shoddy and limited. In part this is a frustration of maintaining objects on Java's terms, but also defects in the JVM: type erasure is a hair-shirt that neither of languages I mentioned previously have to wear. You can somewhat avoid the loss of types with manifest hacks, but it's shoddy business. Often you can't; note that Akka has a distinctly dynamic feel, being that certain messaging functions--!! and !!!--must return an Option[Any] even when the original message is well-typed. The system loses the type information at runtime!

Compilation speed is also an agrivation. It's a slower process than Haskell--not zippy to begin with--and comes with less result, to boot. All that said, I tentatively like Scala, especially if a project is required to run on the JVM. I find it to be an especially pragmatic language, if unbeautiful and not something I intend to use for long-term, exploration coding. A fine choice for a language to be used in anger, let's say, if the memory usage of the JVM is of no concern.

I was very surprised, having only started Scala recently, that it cannot infer types in function arguments. OCaml has no problems with this.

Scala doesn't have Hindley-Milner global type inference (of which OCaml has an extended version of) and instead uses a form of local type inference. Hindley-Milner doesn't work well (i.e. undecidable) in the face of overloading and inheritance.

If I understand "type erasure" correctly, Haskell and ML both do have type erasure.

In Haskell, using typeclasses like Typeable, you get the power of keeping types in runtime, but the types are actually erased.

Oh, sure. Sometimes types disappear at compile-time and aren't needed during program execution: the theorem is already proven so why should the runtime system carry around the baggage? Unless we're talking past one another, the Scala problem is rather worse: start passing around Seq[Seq[Int]]'s through your code and the compiler will be unable to ensure that it's always a Seq[Int] stuffed in there; the compiler comes with warning flags, but it's rather a bother and decidedly more dynamic.

You are quite correct, almost all strongly typed languages perform some type erasure for efficiency purposes. The problem, as I see it with Scala, is rooted in defect in the JVM; the system looses type information and cannot verify at compile-time that all your types will be correct.

Please correct me if I'm wrong or have misunderstood you.

I don't know Scala -- but it sounds like a compile-time type-verification problem more than a type erasure problem.

When Haskell targets x86, the x86 opcodes surely don't help Haskell verify that [[Int]] really has [Int]s in there, but it's not necessary because it is proven at compile-time. So I am a bit confused why Scala needs the JVM's help.

Consider that Haskell does its compilation step once; the types and their parameters are discarded and machine code is generated up front. (More correctly, LLVM IR is generated and combined with the Run Time System into a binary.) In Scala, and Java before it, compilation happens during program execution: you cannot discard types because the compilation step is never over. Ordersky's previous work on Pizza ensured that, however, that type parameters _were_ lost; rather than modify the JVM to support generics it proved to be more expedient with regard to backward compatibility just to strip them out. Pre-generics binary code could co-exist with post-generics because they compiled down to the same bytecode. Hence the complaint that in Java Array[Int] and Array[String] generates two special purpose, 'parameterized' Array structures in memory. Mostly the Java compiler can be clever and produce relatively only what is needed during execution, but the PermGem exception happens because, well, it's a hack.

Now, consider the difficulty of Scala wherein it's possible to represent a compound type. Erasure ensures that only the first parameter in the chain is generated in the type-erasure step, destroying the meaning of the union. There are kludgy ways around this--wrapping, bundling up manifests--but in the end the JVM does not allow the transmission of necessary information efficiently. Manifests serve the purpose of maintaining parameter information at runtime, but do so at the expense of code cache space and stress system resources. PermGem exceptions are much more common, in my experience, when doing Scala work as compared to Java.

The shorter answer to your question is the JIT nature of Java combined with a byte-code designed not to maintain the information needed to generate all possibilities of machine code on demand. LLVM IR maintains such information for Haskell--when targeting machine code directly I believe this is achieved by static polymorphic compilation, but I could be wrong; Stack Overflow time?--and the CLR does the same for C# and friends. It's a JVM problem entirely. The JVM promises to perform any and all compilation but is unable to accept all the information needed to do so effectively.

The Wampler & Payne book "Programming Scala" works an example which is absolutely painful.

They're able to make a language that sorta-kinda looks like plain English, except that it's based on structurally unstable idioms that would require the user to be familiar with all of the corner cases of Scala syntax if they actually want to use it.

That's fair. Indeed, I've found that general purpose programming languages are not the ideal platforms for DSL development (my previous comment about Scala's fitness in this regard should be taken as a relative statement). That is, there are many idiosyncrasies of the host language that leak into the DSL, error handling/reporting is hard and tends to obscure the code, and the eventual DSL tends to have a very hosty flavor.

What we need are special-purpose languages designed for creating DSLs.

That said, I own a whole raft of Scala books, including the original by Odersky, the yellow-black Apress one and so forth; and of all of them, I find Wampler & Payne the most approachable and useful.

If you care about shipping products, it is worth it to care about the tools you use to ship the products with.

Some languages are fashionable because they are neat. Some languages are fashionable because they have allowed people to ship products better and faster. Some languages are fashionable because of both.

There are a handful of commercial success stories with Scala, like the messaging engine behind Twitter, but not a lot.

I can definitely see how an individual or small team could build something fiendishly clever in Scala that's maybe 2,000 lines of code that does the work of 20,000 lines of code in some other language.

On the other hand, I'm not sure that systems 10x that size developed by bigger teams would succeed in Scala, and I fear for what happens when that 2,000 line Scala system falls into the hands of maintenance programmers.

He made his statement in the general sense, and I responded in the general sense. I don't know scala and can't speak to it. But I've heard that exact same sentiment applied to ruby (rails, specifically). But ruby has enabled huge increases in productivity for me over php. So I can safely say that the sentiment that if one is doing something fashionable one must not care about shipping, is full of shit.

    small team could build something fiendishly clever in
    Scala that's maybe 2,000 lines of code that does the
    work of 20,000 lines of code

I actually provoke you to pick a language, any language other than assembler, and write a piece of software with both languages, of reasonable complexity (no micro-benchmarks shit), as concise as you can.

If the source-code in Scala is at least 3 times smaller, I'll buy you a drink.

COBOL?

I work as a consultant in Bangalore; most of my daytime job involves reviewing projects (for code quality). I see about 8-10 projects a month (Java/.Net).

There are projects which are just bad, let's ignore those. Those which can be classified as maintainable, are well structured and object-oriented, they abstract implementations behind interfaces etc.

But, they are verbose. I could probably accomplish the same thing in about 3x less code using FP/declarative constructs.

Reminds me of Yegge's Kingdom of Nouns; it is a funny read if you haven't read it yet.

Yesterday on HN, someone from FourSquare said they have 150k lines of Scala.

http://news.ycombinator.com/item?id=2679041

Is it safe to assume that would be 300k-450k if written in Java?

    There are a handful of commercial success 
    stories with Scala

Correction: There are a handful of commercial success stories with Scala made public. There are companies trying to solve hard problems with Scala that are not necessarily interested in language advocacy.

If you're shipping products and making things of value using great tools (however fashionable they are) it means you spend more time shipping and adding value and less time dealing with the limitations of your tools.

Reply on news.ycombinator.com