Hacker News Reader

> I've watched a company spend two years, dozens of developers, and over a million in ESB license fees failing to write a site that was expected to have, at peak, 100 users/day.

I'm sorry to hear that. But you understand that this had nothing to do with Java, right? Java didn't make those people do that. That was just good old fashioned human stupidity. It's entirely possible to write simple things in Java, without ESBs and a person-century of effort.

It's even possible to write simple things in Java EE - there's an enjoyable little book about it: http://press.adam-bien.com/real-world-java-ee-night-hacks-di...

> Nor are the performance advantages "permanent"; even if you use Java, you will probably have to rewrite if you reach Twitter scale, because your initial app won't be clusterable (or if it is, it's massively overengineered and your business will fail).

I don't think this is true either. The normal approach to writing scalable apps these days is the twelve factor approach:

http://12factor.net/

And that works just as well for tiny deployments as huge ones. It doesn't involve doing anything arcane and expensive up front. It's mostly common sense. Scaling out will involve a lot of changes, but incremental ones.

> I'm sorry to hear that. But you understand that this had nothing to do with Java, right? Java didn't make those people do that. That was just good old fashioned human stupidity. It's entirely possible to write simple things in Java, without ESBs and a person-century of effort.

It's an extreme example of the same mistake this blog post is making. I mean, from a certain point of view they were right - if the site had reached twitter scale, it would have needed those things. The market genuinely is growing, and the problems the system solves exist in almost every industry. I expect those four guys won't keep their system in Rails forever.

Maybe using PHP really will make your system slower by a factor of 15. So what? A trivial saving in development time - say a week over your first year, a 2% factor - is well worth that 15x performance penalty for most businesses.

((Don't use PHP, kids. But not because of the performance))

> The normal approach to writing scalable apps these days is the twelve factor approach

Hmm, I wonder how many people have heard of this, and you say it's "the normal approach".

Debugging distributed systems is and always will be harder than debugging monolithic systems. Even the simple engineering good practices from that page, like explicitly declaring dependencies, have a cost. My Google friends tell me their explicit policy is to design a service for one order of magnitude more than the current traffic, and expect to rewrite it after that.

And once you're doing the things you need to scale - explicitly defined interfaces, all calls going via the network or something equivalent to it, service discovery - then changing the language used in parts of the system really isn't that big a deal. You can do it incrementally. (I have experience of doing just that, at last.fm)

> I expect those four guys won't keep their system in Rails forever.

For people who didn't know this, I would like to add that Twitter was written in Rails as well. They switched later on for a couple of reasons[stackoverflow].

[stackoverflow] https://stackoverflow.com/questions/9747857/why-couldnt-twit...

i use PHP and i got customers and revenue and retention rate too

facebook use php

deal with it

programming language is one thing, marketing your solution is another thing

facebook is using php compiled to C++, and that's only for their frontend. Mind your anecdotal evidence :)

Actually, Facebook is using PHP executing on a JITting virtual machine:

https://www.facebook.com/notes/facebook-engineering/speeding...

Mind your out-of-date rebuttals to anecdotal evidence :)

It's entirely possible to write simple things in Java, without ESBs and a person-century of effort.

That I don't disagree with (see Java4K and various other demoscene-like competitions), but when the mainstream culture of Java is oriented towards architecture-astronautism and "enterprisey" bureaucratic excesses of complexity, that's what the majority of Java developers you find are going to do. Ask them why a simpler (and less time-consuming) solution wouldn't suffice and they'll respond with things like "robustness", "extensibility", "modularity", "encapsulation", "abstraction", etc. even when none of these properties are truly applicable to the situation at hand. If I were to guess why, they're just so accustomed to writing code with such indirectness that a simple solution feels "wrong" to them. I worked very briefly for a Java shop and on one occasion my code was rejected, and was told to "refactor" a fairly trivial 20-line method in one class into 8 separate classes each containing 1-line methods on account of the fact that my original code was not "extensible" nor "modular" enough. I was suggested to introduce a design pattern to do this. Any advantage of that extra flexibility was never realised since the application was eventually completely scrapped in a rewrite. (I left shortly after that.)

"was told to "refactor" a fairly trivial 20-line method in one class into 8 separate classes each containing 1-line methods"

How typical is this level of non-value adding misguided pedantism? I understand that big software consultancies would want to milk their clients for all the work they can for implementing function points. But what profitability models drive developers to this level of weirdness? Is it just a cultural pathology?

This is excessive but I've seen numerous similar cases. Premature code future-proofing has always struck me as similar to premature code optimisation. Generally something that newer/weaker devs tend to do until they learn better.

It's plain and simple pedantry, and it's not confined to Java. This mindset has a healthy representation in the Ruby community, which actually makes programming in Java much less distasteful to me -- if I'm going to be writing AbstractSingletonFactoryAdapters, then at least let my AbstractSingletonFactoryAdapters be fast.

Enterprise. It is just how the enterprise world works.

No it is not. Just because you have a one bad experience, doesn't mean that is how the enterprise works. There are hundreds of thousands of enterprise shops out there. We don't do anything like the craziness you guys are talking about. We make it work with the easiest code possible. If necessary we refactor when we need to make a solution more generic.

Java at Amazon, Google, Twitter etc. is generally less pedantic and more focused. I have been a Java developer since 1997 and the only time I've seen the type of pedantry you are talking about is at non-technology companies where the most important metric is how cheap a developer is, not how much value they can add.

Is there an update to this book or something similar that speaks to Java +7?

The book is mostly about the use of the Java EE APIs. As such, it's not all that affected by the evolution of the core language from 6 to 7 to 8.

However, it probably will be affected by the evolution of the Java EE API. EE 6 was a huge, massively simplifying, change from EE 5, which is why it needed a book like this. EE 7 was a fairly incremental change from EE 6, and EE 8 is set to also be a modest step forward.

The biggest change in EE 8 - and i don't know whether to laugh or cry about this - is that Java will be getting a standard web MVC layer. Fifteen years after such things first appeared on the scene, and five years after they became passé. Maybe there'll be an API for controlling dialup modems or AIM clients next.

> The normal approach to writing scalable apps these days is the twelve factor approach

I've never even heard of this. I agree with most of this advice, but, I wouldn't exactly call it a defacto standard.

My bad. I suppose i live in a bubble just like anyone else. I do think this is a well-known approach within the bits of software culture that talk about scalability. If you're a regular reader of highscalability.com, or a developer at Netflix, or one of the sad nerds who wish they were (eg me), or you've read the Heroku documentation, you've heard of 12-factor. But no, not everyone has, and in particular not every working stiff who shovels Java for a living.

> they can't make the code as easy to comprehend as it would be in a more expressive language.

It's a little hard to make a general statement using an vague term like "expressiveness".

Static typing can actually make things clearer. That's a kind of expressiveness. You can know with complete certainty what type of data the code works with, just by looking at it. No need to rely on a certain set of unit tests living somewhere else to confirm your assumptions. And if you say "well just read the function docs", how is that any less verbose in the end? It's the same verbosity, minus the compiler guarantees.

On the other hand, there are situations where DSLs can greatly cut down on route programming. Like Apache Pig vs. calling Java APIs. But that's a more extreme example, an actual speciality "domain" to make a specific language for.

I might help to use a more precise term like verbosity. Then we can ask, is this verbosity really helpful (like explicitly declaring types), or can we compact it without losing significant information. Fortunately Java 8 has made some progress with adopting features that cut down on unnecessary verbosity.

Static typing can make things clearer, indeed, however Java has really shitty static typing. Also this is a common misconception:

> You can know with complete certainty what type of data the code works with, just by looking at it.

First of all, the type is only a limited view of the shape of the underlying data, written from the perspective of the business logic using it. In a micro-services architecture it happens often to see the same data exposed with types that are often very different, depending on what you're trying to accomplish. And too often developers that work with Java are complecting the business logic with the actual data, which is often pure data that's being messed up with short term business logic rules.

And furthermore, Java developers don't work with static type safety. Tell me, what does a String tell you? What does a Double tell you? How often have you seen explicit type-safe numeric types in Java (e.g. lets say values that are in kilowatts, or meters, or other units of measure)? You don't see that, because Java is awful at expressing such types, from the lack of operating overloading, to the completely retarded generics system and the lack of type-classes.

Second of all, the type IS NOT sufficient documentation when dealing with non-determinism and if side-effects happen in your system, then that's non-determinism. And when non-determinism happens, you need to document the protocol of communication and static types in mainstream languages cannot capture that, which is a PITA because that would be far more useful than seeing that this Person has an email address.

> In a micro-services architecture it happens often to see the same data exposed with types that are often very different

You mean micro-services that use dynamic instead of strongly typed APIs? That's really more a critique of such architectures than the facility of static typing.

> Tell me, what does a String tell you? What does a Double tell you?

It tells you you can't use a String where a Double is needed, without unit tests, from the very moment you type the reference in your IDE.

You're correct that it doesn't eliminate all possible problems such as validating the unit of measurement, and that it's not sufficient documentation in and of itself. But that's not the goal of static typing. It's there to eliminate a very large swath of problems through simple declarations in code.

Type safety is valuable, but you can have equivalent or better type safety in a much less verbose language than Java. e.g. I use Scala - it's not quite as concise as Python, but it's pretty close, and the language is powerful enough that I can express more possible effects directly in the type system (Does this method make an async call? Does it access the database? Does it log audit events?) without the code becoming unreadably verbose.

Care to share more details of these companies including how the Ruby ones become profitable after a month with 4 guys on payroll? No offense but I've got to check the facts these days.. :).

Also, don't forget Fred Brooks article about the Second System. I betcha smart engineers on tight budget can hack in Java just as quick as those 4 guys and achieve the same result. My apology, Java today is as productive as Ruby on Rails.

The difference between rewriting a Rails site to Java and an already written system un Java to a more modern architecture is huge. One can slowly move from old Java architecture to the newer one but it'll be trickier from Rails to something else... don't forget the accumulated Ops experience managing jvm based app server, the devil in details for those experienced engineers who have been there and done that...

> Care to share more details of these companies including how the Ruby ones become profitable after a month with 4 guys on payroll? No offense but I've got to check the facts these days.. :).

It was small industry, those were basically the only two companies in it, I don't feel I can. (I've already changed some small details e.g. it wasn't actually Ruby). In any case it's just one person's anecdote - but I bet it's happened a lot.

> My apology, Java today is as productive as Ruby on Rails.

Citation needed. In the programming language shootout Ruby still comes out much shorter than Java, and like I said, the only proven risk factor for bugs is number of lines of code.

> One can slowly move from old Java architecture to the newer one but it'll be trickier from Rails to something else...

When I worked at last.fm this was literally my entire job (well, from PHP). You're rearchitecting anyway, creating stable interfaces and putting load balancers in there because those are also things you need to do as you're scaling up. Changing languages is no harder.

> don't forget the accumulated Ops experience managing jvm based app server, the devil in details for those experienced engineers who have been there and done that...

You can hire the experience - Java is literally the easiest language to hire for (I mean, hiring is still hard, and there are a lot of crap java programmers around, but there is a bigger pool of experienced programmers out there than any other language). And you're going to need to hire anyway; your ops team will have to grow, your engineering team will have to grow more, and you absolutely cannot rely on the engineers who bought the first version still being around when the time comes to rewrite; the half-life of an engineer in a startup is not large.

Twitter was the article's choice of example, not mine - and from a business point of view Rails worked really well there.

>the only proven risk factor for bugs is number of lines of code.

I'm sure you can cite that, but it feels wrong in this context.

That is, given the same language, or equally verbose languages, it may be true and applicable. But, if comparing a more verbose language to one that is less so, then the former will naturally yield a higher LOC count. But, does this mean it's more bug prone? There aren't any more logical constructs, flow control mechanisms, etc. The language simply requires more lines to express the same intentionality.

In fact, I wonder how a more verbose language would compare with a less verbose one if we normalized for some verbosity factor. So, let's say language A is twice as verbose as language B, such that when the LOC count of A is twice that of B, we consider them equal (call it the same "Effective LOC").

I actually wonder if, in this case, language B would have a higher bug count per effective LOC, owed to its terseness and/or reduced readability.

>You're rearchitecting anyway, creating stable interfaces and putting load balancers in there because those are also things you need to do as you're scaling up. Changing languages is no harder

I have to disagree here. With just a little forethought, a good bit of the code can be reusable. For instance, your data access layer.

And, the more forethought you put into it, the larger the potential reusability. This doesn't have to be premature optimization. It's just a balance between getting it done as quickly/efficiently as possible while still making some sensible, fairly low-cost coding decisions for scalability. For instance, everything I write now is load-balancer ready. This was achieved largely through a simple library that understands its environment via a simple config directive, and proxies relevant calls (e.g. when pulling the client IP or checking whether the request is secure, it determines when to pull from the request headers stamped on by the load balancer versus invoking request.getRemoteAddr() or request.isSecure()). Of course, there are other coding decisions you make, but it's really not as much extra work as how you implement.

In any case, unless you're purposely writing completely nasty throwaway code, there is always some reusability, which typically makes starting from scratch in a new language a much weightier prospect than making some scalability enhancements to the existing base.

> In any case, unless you're purposely writing completely nasty throwaway code, there is always some reusability, which typically makes starting from scratch in a new language a much weightier prospect than making some scalability enhancements to the existing base.

It's not really from scratch. If you've figured out the correct structure and interfaces to solve your problem you can reuse all that when porting to a different language, and that's the hard part of development.

>If you've figured out the correct structure and interfaces to solve your problem you can reuse all that when porting to a different language

Well, I think you're underestimating the sheer brute force work needed to port an app with a large-ish LOC base.

But, beyond that, if you've gone through the care of structuring it correctly such that the structure and interfaces are preservable once scalability needs arise, then all the more reason not to port it. Presumably, at that point you'd have, not only the structure, but working code.

So, if it can be so readily adapted to a new language and you're preserving so much code, then why bother porting it? Kind of makes may original argument. Just apply the scalability tweaks on the existing code base and you're done.

> Well, I think you're underestimating the sheer brute force work needed to port an app with a large-ish LOC base.

I have done this professionally. You don't port everything, you just port services as they become bottlenecks. By the time you need to scale like this, you already need to have multiple engineering teams and you need well-defined interfaces between components, so having parts of your system in different languages doesn't impose an additional cost.

> So, if it can be so readily adapted to a new language and you're preserving so much code, then why bother porting it? Kind of makes may original argument. Just apply the scalability tweaks on the existing code base and you're done.

Sure, which is probably why Digg was happy to carry on using PHP. For the vast majority of systems you never get to the scale where using Java is worthwhile (at least in performance terms).

When do you want to port? When your server costs get high enough that a 15x performance factor is worth the constant cost in dev time that using a lower-level language would cost[1]. It might take more effort than writing in Java from day 1 - but it might not, if you go through several iterations of prototyping/refactoring in PHP and then only port once you've found the correct design. And more importantly, you get to pay the Java price much further down the line, when you're bigger and more profitable. As PG says, an early-stage startup should do things that don't scale.

[1] I'm not convinced you can't have your cake and eat it, with e.g. Scala or Haskell. But assuming for the sake of the argument that the only languages were Java and PHP and PHP really was more productive than Java but less performant.

>I have done this professionally. You don't port everything, you just port services as they become bottlenecks.

Yep, been there too. Except, in the case I'm thinking of, it wasn't for performance. We went with a multi-language approach for different reasons, and we ended up with a hybrid that no-one liked. Then, we finally decided to port everything to one language.

All of this to say that a multi-language approach can have its own drawbacks, which is not to say that it's never appropriate. There should just be a really good reason for it. Generally, I'm not sure that starting with a plan to half-port a non-performant app is a good reason. But, I'd never say never.

I'd also add (again) that if you've really taken the time to structure the app so well that you have clean interfaces and modularized, easily-portable services, then it begs the question: did you really save so much time by then implementing that design in PHP vs Java? As you said, the design is the hard part, and I'm just not seeing the notion of a quick non-scaling app, somewhat hastily implemented in PHP, being compatible with the idea of a tight, service-oriented, modular design. I think you kind of have to choose one.

I don't suppose any approach is necessarily the absolute right approach per se. But, given that the article is about picking one approach/language (Java) and sticking with it, I'd have to say that I'm still convinced that it may indeed be optimal, even after reviewing your points.

> I'm just not seeing the notion of a quick non-scaling app, somewhat hastily implemented in PHP, being compatible with the idea of a tight, service-oriented, modular design. I think you kind of have to choose one.

Oh sure, the initial code was pretty terrible. But by the time we were moving parts to Java it had all gone through a lesser or greater amount of refactoring; the best design is iterative and emergent, and we had the http://c2.com/cgi/wiki?WhatIsAnAdvancer experience several times over. In the early stages you really can iterate faster in not-java; as the correct design emerges and the codebase gets larger the interfaces tend to firm up, and at that point it becomes more reasonable to port pieces for performance.

> like I said, the only proven risk factor for bugs is number of lines of code.

There's never been a proof that writing code in Ruby WILL give you LESS bugs than in Java just because you'll end up with LESS LOC. I think this is a generic statement that needs to be clarified.

I agree and I will not argue that LESS lines of code "might" give you LESS bugs given that you're writing it using the SAME language (version 1 in Java has more lines than version 2 in Java, maybe because in version 2, the developer gained more experience and can write idiomatic Java and use libraries instead of rolling his/her own). But Ruby yielded less bugs because of less line of code is a marketing speak to sell the religion of Ruby.

Sometimes I wonder why developers aren't forced to just use LISP since everybody in the whole world knows that LISP is super expressive that it MUST resulted in far less code right? Right?

> When I worked at last.fm this was literally my entire job (well, from PHP). You're rearchitecting anyway, creating stable interfaces and putting load balancers in there because those are also things you need to do as you're scaling up. Changing languages is no harder.

Fred Brooks Second System http://en.wikipedia.org/wiki/Second-system_effect is exactly what you're experiencing, forget the language war for a second please. Really.

I work for a company that has been using Java since probably 2002/2003 (I joined in 2012). Server-side in Java with Client-side in Swing, slowly evolving the Server-side POJO (probably no Application Server), to EJB 2.x (shudder) to EJB 3.x and goes to the cloud we go (AWS). Recently we refactor a certain part of our code and added Microservice written in JAX-RS because we have to integrate with another product suite internally. That's what I called re-use and evolution of code-base: 10 years of Java where some of the code from the past were/are reused and still in production today. (.... but boy those old code aren't pretty, hehehehe).

Oh, I almost forgot the part where we used shell script to build, moved to Ant, moved to Ant + Ivy, and slowly moving to Maven (if we need to write a new microservice, maven by default, the rest have to move slowly).

Oh, here's the best part: last year we moved away from homegrown migration tool to use Flyway, 4-6 lines of Java code to initialize and execute the migration workflow dropped in our old Java code.

So yes, if we had to re-write and re-write and re-write to different languages every time, it is definitely will be harder.

> You can hire the experience - Java is literally the easiest language to hire for (I mean, hiring is still hard, and there are a lot of crap java programmers around, but there is a bigger pool of experienced programmers out there than any other language).

Ops, not devs.

> and you absolutely cannot rely on the engineers who bought the first version still being around when the time comes to rewrite; the half-life of an engineer in a startup is not large.

Thank God it's Java, not PHP, not Perl, not Ruby (I give Python a pass because I'm being nice). IDE helps a lot. This reminds me of Zed Shaw blog about charging a lot of money fixing Rails mess as a consulting service.

> Twitter was the article's choice of example, not mine - and from a business point of view Rails worked really well there

I'm not suggesting that they made a bad move or whatnot I'm just merely debunking the common stories of how small team beat big team (that may have big expectation/big scale yet not focused on solving customer needs) because of the technology/languages. Talent is something else. Tech/Languages/Scaling? sounds like ego problem there.

> Sometimes I wonder why developers aren't forced to just use LISP since everybody in the whole world knows that LISP is super expressive that it MUST resulted in far less code right? Right?

You've fallen for Lisp propaganda :P. If you look at the shootout, Python and Ruby are comfortably ahead of Lisp in the expressiveness stakes.

> Fred Brooks Second System http://en.wikipedia.org/wiki/Second-system_effect is exactly what you're experiencing, forget the language war for a second please. Really.

Really not. This was one of the leanest places I've ever worked; everything we did there was absolutely necessary for the scale we were operating at. I've seen second-system effect in action; last.fm wasn't it. Writing Java will not save you from having to scale horizontally; maybe it's 2x, 5x, 10x faster than a dynamic language, but if you're growing then these kinds of constant factors only buy you a small amount of development time.

> Recently we refactor a certain part of our code and added Microservice written in JAX-RS because we have to integrate with another product suite internally. That's what I called re-use and evolution of code-base: 10 years of Java where some of the code from the past were/are reused and still in production today. (.... but boy those old code aren't pretty, hehehehe).

If you're using microservices multiple languages really shouldn't be a problem. It's not like we rewrote everything; services that were fine in PHP stayed in PHP. We moved things to Java as and when we needed to. If everything's a service (Thrift in our case) then it really doesn't matter what language is on the other end.

And it's not exactly "from scratch". Physically typing out the code is never the bottleneck; most of the hard part of programming is design, getting the correct decomposition of the problem. Porting well-written code from one language to another language that supports the same paradigms is really not that hard, and certainly doesn't mean throwing away all the lessons of the first codebase.

> If you look at the shootout, Python and Ruby are comfortably ahead of Lisp in the expressiveness stakes.

The code in the 'shootout' is verbose, to make it fast. Much faster than Ruby or Python.

Generally small code in Ruby and Python can be more compact. For larger code this is then no longer the case, once the macro-advantage kicks in...

> Verbosity matters for reading and maintenance far more than it does for writing.

I would argue that on average verbosity improves readability. Yes it may take you slightly longer to read the code, but verbosity often results in the code conveying more information or expressing it clearer (or both).

Just repeating the same thing twice doesn't help with anything. The standard examples of Java verbosity - repeating variable types, getters and setters, anonymous classes instead of lambdas, using the visitor pattern to emulate pattern matching - don't make anything more readable or clearer (indeed many people find the Java version more confusing to read).

What features does Java have that you're saying help code convey more information or lead it to be expressed more clearly? I find that my more concise Scala code usually both is clearer (there's just less irrelevant clutter e.g. braces) and conveys more information (because e.g. error handling with Either and for/yield makes it clear which parts of a function could have errors, whereas with try/catch all you know is "there's a function somewhere in this block which throws").

> Just repeating the same thing twice doesn't help with anything.

I completely agree - but not all (or even most) of Java's verbosity is from repetition. The traditional example of `Foo foo = new Foo();` as repetition ignores the fact that `Foo` will not always be repeated as the first will often be an interface (or superclass) and the second will be an implementation, e.g. `Set<String> set = new HashSet<>();`

> What features does Java have that you're saying help code convey more information or lead it to be expressed more clearly?

My comment regarding verbosity wasn't in comparison to Scala, I was thinking of Ruby, Python, Lua, et al (I'm a huge fan of Scala btw, but the post was on the benefits of Java versus dynamic languages). Anyway to answer your question with a few examples:

* Java's method signature gives me a lot of information, looking at `public int doStuff(String foo)` tells me the visibility, what it will return, and exactly what I can pass it - even to someone completely unfamiliar with the language. Looking at `def doStuff foo` tells me nothing other than how many arguments it takes (if you're familiar with Ruby you'll also know the visibility).

* Similarly when you see `return foo` you know (even knowing nothing about Java) that it's returning `foo` and that `foo` is the intended return value.

* Someone who's never read a line of Java in their life can look at `Map<String, Integer> foo = new HashMap<>();` and know that `foo` is a map (and will most likely conclude it has Strings for keys and integers for values). If you've never coded in Lua and you see `foo = {}` you have no idea what that is (if you've seen similar in other languages you can make a good guess, but you don't know).

I would rewrite that as "_explicitness_ improves readability". When I read Java code, I rarely have trouble to know what the code is doing (at least at a local scale).

Not all static-typed languages are readable because they're not all explicit. In C++ for example, even the most trivial snippet of code can be obfuscated by all sorts of abuse: operator overloading, implicit calls to copy constructors, preprocessor macros, etc. Scala suffers the same problem, perhaps even worse, doing tons of stuff automatically i.e. in a way that's not obvious by just staring at a code fragment such as an individual method. Organizations that have success writing large-scale code in such languages [I can speak for Google here], typically have careful style guides that ban the bad stuff and are strongly enforced by code review process; but in that case you're not using language X anymore, you're using some "safe subset" of X.

Of course you can't have any high-level abstraction without some tradeoff of maximum explicitness; virtual calls for example make hard to know which exact code is invoked by "just staring" at an invocation. The trick is striking a healthy balance between power and readability. Java is not perfect in this criteria (it errs a bit in the side of caution), but it's certainly much better than any dynamic-typed language.