How We Went from 30 Servers to 2: Go | Hacker News Reader

How We Went from 30 Servers to 2: Go

blog.iron.io

688 pointstreeder495 comments

It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made.

frameworks on top of frameworks, all being over engineered, with poor understanding of what the system actually does, result in super slow apps on top, that you generally go to aws to scale.

It's not the first time that I see people reducing a dozen servers that were "always maxed out" by a couple of servers "that can barely feel the load".

Yeah throwing hardware at the issues is fine'n all but we're been way over the limit too many times in too many directions.

Go provides a clear start/APIs/framework, and the language enforces good habits. Also, it doesnt have things like global interpreter locks.

The lesson? Stop using cool techs because there's blogs about them. UNDERSTAND the tech before using it.

I'm very sympathetic to your tech-on-tech-on-tech-on-tech hogwashery promulgated by open sourcers & vendors prosteltyzing their wonderful solutions and the other acculturation factors that permits developers approaches other than trying & seeing & exploring-

That said, you are wrong.

I'm not a Go user and I don't care for it, but Go is fundamentally different and better than most runtimes. Go has it's own green threading, which allows it to start new tasks using extremely small amounts of memory, and it can switch between tasks without the normal cost of context switching full on threads.

That is a huge difference. Node has some elements in play here, with it's event loop and callbacks, but really the only thing in the field that compares to Go that's in use is Erlang, and there's no frameworking on top of Erlang that wouldn't cause most PHP or nine tenths the Python developers to curl into a weeping ball at the Eldritch horrors they'd been exposed to: even ASM code is not nearly so effective, being clearly a low level somewhat harmless thing.

Being able to kick off small processes, that can sit around for a really long time, awaiting data- that's a model of computation we haven't done in decades, never with the mass popularity- there's a TLA for this kind of processing- Communication Sequential Processes, and Go is it. It's entirely different from the procedural model, from the Python and Perl and PHP scripting, where concurrency is a carefully waded into thing- Go is a concurrent first runtime, about ongoing, enduring, concurrent processes, and that's huge, HUGE I say, huge.

> I'm very sympathetic to your tech-on-tech-on-tech-on-tech hogwashery promulgated by open sourcers & vendors prosteltyzing their wonderful solutions and the other acculturation factors that permits developers approaches other than trying & seeing & exploring-

Are you studying for the SATs, or just trying to make your argument sound more compelling through the use of unnecessarily complex words?

Ia, Ia, Erlang Ftagn!

I agree though, I just wish that Go would have developed from Erlang as a runtime.

The Erlang virtual machine and systems are beautiful things - I used them for some wire-level work on a project and bitfields are awesome, and it provided a clean interface to a more 'normal' language to display to the user, fail-fast fault tolerance on a public safety project, neat stuff.

It's just erlang-the-programming-language that has all the Prolog warts that scares people, I think.

Correct me if I'm wrong, but the Haskell concurrency model is a very efficient one which is also based on lightweight threads. Moreover, is CSP is all you care about, you might want to have a look at CHP.

Go is... fundamentally different than most runtimes? Probably. Better? Maybe. Better than the comparable ones? I think it's losing that war. You don't even need a special runtime, it can help, but I think the amount it helps is overstated in Go's case.

Concurrency isn't that hard in practice. If you're willing to forgo automatic multiplexing Perl will give you Go's concurrency model in a library written ten years ago, and it works as naturally as it does in Go. Python seems to have similarly good support but it probably doesn't look much like Go. I'm not sure what Lua's doing these days, but in the past I've found concurrency dead simple. If you happen to be writing C, time-slicing the work isn't even that hard. If you know enough about Go's concurrency to tell whether a single goroutine will cause its scheduler to deadlock all by itself, you can easily write concurrent programs in all these languages.

I know this will be a controversial claim, but all of these solutions are easier to reason about than Go. Usually not by much, and it has primarily to do with the nature of its multi-threading support, but that's just the concurrency model being weakened by the support for convenient parallelism.

There is a case where Go is best: when you like writing Go more than the others. You can guarantee a little less about concurrency than most, but at least you can usually guarantee more about types than most. If you like writing Go you will either luckily avoid the problems or develop habits that prevent you from producing concurrency bugs, possibly even without realizing it.

You probably stopped at the half of my comment, because I did write Go had advantages (over the likes of ruby&friends anyway).

But it's the wrong way to think about it, that's why it's in the second half of the comment. Else, people will just use go because they're told it's the cool kid on the block. WRONG way of thinking.

That is all :)

You could have linked to Occam[1,2]...

[1] http://pop-users.org/occam-pi/

[2] https://en.wikipedia.org/wiki/Occam_%28programming_language%...

They went from a language with a factor of forty cost compared to C++ to one with a factor of four cost. They saw a factor of ten improvement in CPU utilization. Seems like all Go really brought to the table here was speed. I don't see any evidence that its unusual features were a factor, besides its base-level increased efficiency.

Nice rant, and I mostly agree with it, but there are a few slow things about the ways that some languages are usually implemented that don't really have much to do with over-engineering. Let's take this line of Python code, for example, and look at how the CPython interpreter runs it:

    return x + 42

This turns into the following bytecode instructions:

    LOAD_FAST       0 (x)
    LOAD_CONST      1 (42)
    BINARY_ADD
    RETURN_VALUE

First we get the x variable from the local namespace. This is an array of PyObject pointers; the LOAD_FAST oepration simply does an array lookup and puts the result on the stack, incrementing the reference count. Pretty fast. Next is LOAD_CONST, which is even faster; it takes an already-allocated PyObject and puts a pointer to it on the stack, incrementing the reference count. BINARY_ADD removes two numbers from the stack, dereferences the pointers to get the integer values, allocates a new PyObject with the resulting integer inside, bumps up its reference count, decrements the reference count of the two operands, and pushes the result on the stack. Finally, RETURN_VALUE jumps back to the caller of the function.

In Go, the corresponding code would be compiled to two, maybe three machine code instructions. There are good reasons why Python does it this way, but it does suffer some inherent slowdown.

A more interesting comparison would be the PyPy-generated native code for the expression versus the compiled code from Go. Comparing an interpreter with a compiler is generally uninteresting; the compiler almost always wins.

If you look at the code that comes out of various modern JITs you often will find really interesting differences in the native code they produce, even for simple constructs like arithmetic. Type checks, barriers, bailouts, etc - in addition to more mundane differences, like one using SSE to do floating point and the other using x87, and one JIT having a better register allocator.

> dereferences the pointers to get the integer values

Doesn't it find a method of x that implements addition?

> It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made.

Exactly.

For the type of product they sell, if the processing time isn't pretty much all going to the actual running of the customer supplied job code and network latency, they're doing something very wrong. The language choice shouldn't be much more than noise in a setup like that. The framework choice, on the other hand... Rails is hardly well optimised for a high traffic API that can't readily be cached.

I've done queuing and job processing in Ruby. Spending 90% of the time in kernel space handling network traffic or waiting is not hard to achieve for a pure queuing server. If you add delegating jobs to external code to do the actual work, you should spend less time in userspace in the code actually processing the messages.

If language choice nets you more than 10%-20% in a setup like that, something else has changed beyond language.

And that's fine. But a post focusing on the language change then makes it seem like they didn't understand what caused the performance problem in the first place.

Maybe I'm just a new and terrible programmer, but I feel like if you wait to fully understand whatever tech is out there in terms of what to use, you'll never get started.

Let's be new and terrible programmers together, because I feel like the only way to fully understand whatever tech is out there is to actually use it.

In reality you have to follow the buzz in the tech to understand new technology. If something sounds like you could use it, take a few days out to toy with it. If you still want to use it, try it out for a small project and the rest will come automatically if it is the right choice.

> It's not really "go" that makes the difference

But from what I've seen and read about Go, it does seem to have things in it that do make a difference.

For example from day one Go was designed for with concurrency in mind and that's a big plus.

To see how this can help this Rob Pike video does a good job of showing off Go's version of concurrency:

http://blog.golang.org/2013/01/concurrency-is-not-parallelis...

Now, you can implement concurrency lots of different ways using many different languages. But I've used a lot of languages and I never seen concurrency done as easily as shown in that video.

> But I've used a lot of languages and I never seen concurrency done as easily as shown in that video.

So I take it you never used Erlang.

Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go.

We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far.

Lastly, for anyone thinking about taking the plunge, use Go tip (from their source code repo) - don't use their "stable" releases. They fix bugs so fast you'll always want to be current.

Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.

Sure, check out their github pages (here's a few):

* https://github.com/mozilla-services/heka-mozsvc-plugins

* https://github.com/mozilla-services/heka

My friend works for Mozilla - that's how I knew.

Rust and Go are not similar or competing. They are both new, but aside from that the differences are huge. So if a project uses one, that doesn't mean the other was a possibility for that project too.

Just curious, what were the tradeoffs for profiling the Python code and rewriting slow parts in C vs total rewrite in Go? For high level languages, the usual argument has been to rewrite just the slow parts in C or some other low level language.

And that's what we did during our first pass: identify the slow parts and port them to C. This gave us about 30% better performance. Without getting too much into the nitty gritty of our specific case, this wasn't enough.

We needed some massive speed improvements, I'm talking in the order of 100X faster. The nature of our algorithms was such that they could be done in parallel (i.e map/reduce) - an ideal candidate for Go's goroutines. We actually tried to make it parallelizable at first in Python, using gevent (and even just multiprocessing) and the results were not great.

One other aspect that really guided us towards Go was memory usage. Python was just sucking up so much memory whereas our Go implementation thus far is so much thinner.

One thing you don't get in the C/Python combo is total static typing. The other big win of Go is elimination of a lot of runtime errors.

So it comes down to which will be more of a win for you and your product. Go has a big advantage here IMO though because of static typing AND performance increase.

Hi, I'm in the market* for an Arabic sentiment engine for some academic research, can you contact me? My email is in my profile.

*Note that "in the market" doesn't mean I have a lot of money to spend :D

What is an "Arabic sentiment engine" :-D? Like textual analysis as to whether a piece of Arabic text is positive or negative?

I went briefly to through the comments, it seems that no-one is bothered with lack of details in this post, things are not logical at all, almost like it was written by PR agency in charge of promoting Go.

Except it is established business so I would assume things are true, but real motives are hidden. Go is obviously good language for some specific things, but it's not like ruby is pure trash, how come you did 'everything' in ruby, and then didn't know about eventmachine, but just had to rewrite things in Go.

And rewrite happened overnight, right?

A lot is missing from this story. I will definitely look more into Go, mostly because someone compared it in comments with TurboPascal, and I have fond memories of Borland tools, Pascal especially.

Go and Rails are so different that there is almost no point in comparing them.

> but real motives are hidden

Could you elaborate on this please? Are you referring to the OP's motives of publishing the article? Or of switching to Go?

> Go and Rails are so different that there is almost no point in comparing them.

Except when one solves the same problem better than the other.

- They did not say why they did not choose one of the other alternatives.

- They did not say how they used the concurrent features of Go, only mentioned that they were there.

- They did not say how long it took to rewrite.

- They did not say what they changed in the API.

I'm not saying the story isn't true, but for a true story it lacks a lot. You could summarize the article with the title and you won't be missing much.

I was referring to switching to Go, I am not disputing languages performance, just the whole article is very strangely written

I was originally very excited about Go when I first learned about it. But then I got tired and frustrated quickly after having to listen to the other Gophers telling me that I don't need this or that feature because there is a better way to do it in Go.

Like.

I don't need exceptions because Go function can return multiple values.

I don't need a mocking framework like Mockito because Go has interfaces.

I don't need an interactive debugger because I can debug with command line using gdb.

I don't need named arguments because I can instantiate a struct and call my function with it. (Have you seen Ruby or Javascript code? Almost every function takes 'opts' as a single argument. Go is probably going down this path too.)

Then I learned about Scala. I'm not saying that Scala is better than Go. However, it has everything that I need. :)

Comments like this are weird. Go does not like exceptions. They've built a whole idiom around error returns. When you said, "I need exceptions", what did you expect them to say? "Oh, sorry, we forgot that, we'll get right on it"?

I also don't understand what you mean by "interactive debugger". What is gdb if not an interactive debugger? Gdb is my go-to debugger for Ruby as well.

Something like RubyMine IDE debugger or Firebug javascript debugger.

If you are a Ruby developer then I highly recommend trying out RubyMine. I promise that you will never debug Ruby code using gdb again!

> Gdb is my go-to debugger for Ruby as well.

Unless you are debugging the ruby process(and not the running script), what does gdb buy you over ruby-debug?

It's pretty typical of the Plan9 mentality. See this post on the Acme text editor: http://9fans.net/archive/2008/08/134

Is that a parody?

wow, worse than the irc python channel.

We also checked out both Go and Scala and picked Scala due to better IDE support (IntelliJ) and existing Java lib ecosystem. It also is typically faster than Go (but does use much more memory): http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

Overall very happy with our choice. I will probably learn Go too, but I also don't really like the way it does error handling and the tooling support seems limited vs what I can get for the JVM.

Yup. This is my exact conclusion too.

Scala fan here. But still, I can sympathize with Go developers on getting rid of exceptions.

I don't know if Go would quite feel the same, but Scala's Try, Pattern Matching and Curried functions are stupid nice and make handling errors without exceptions feel very natural.

Took me a couple months to get there, but there you go. "Folding" out different state feels like I'm writing more robust code. Where I'm coding for a range of possibilities instead of just the golden path.

Even when I _am_ focusing on the Golden Path, features like Try[A] force me (well, warn at least) to at-least stub out the other options so I get to feel like instead of having to hammer in robustness, it's something composed instead. I can focus on making the ideal X, and handle the possibility of needing a notX as alternative realities, instead of trying to build a cyborgXYZ that can handle every potential state. If that makes any sense.

Same thing happen to me, I even did some small contributions around 2010.

Then I discovered D and Rust are better languages for my purposes.

Just a guess, I'm guessing python isn't your favorite language either? They seem to be rather unopinionated languages that let you have access to lots of tools. I hear they manage to keep it pretty simple despite having a "kitchen sink", so definitely interested in both.

D syntax is nicer than Go's, but its also dead compared to Go. Where are you going to put your chips?

tl;dr: I tried Go, but it wasn't like Ruby, so now I'm trying Scala.

Ha. Is there a high performance, compiled, and concurrent friendly language that looks like Ruby? Please share. :)

For large projects, it'd probably make more sense to rewrite the critical code sections in C. rubyinline makes this really easy, and ruby C extensions aren't that bad either.

One time on a contracting gig I had some code that was interacting with cairo drawing and imagemagick. They didn't have a compatible raw image format at the time (one was RGBA and the other was ABGR, IIRC.) It's trivial to convert between the two in ruby, but it was taking 60 seconds per image. Once I switched that one loop to C, it took 0.02 seconds.

I've got nothing against Go, just suggesting an alternative that might make more sense in some situations.

This definitely does make a lot more sense than throwing out your whole app and rewriting it in many situations, but I'm always a little hinky about moving stuff into C just because it has so many ways to shoot yourself in the foot. You can obviously create bugs in any language (I know I certainly have!), but Go really does hit a nice spot between safety, productivity, and speed, so I can definitely see the attraction.

For web servers there are other things more important than the raw processing speed of a single routine. What you want is to get the CPU usage to 100%, CPU resources used preferably in actual processing and less on things like garbage collecting or blocking for I/O. For web servers in particular it is tricky since processing requests also involves a fair bit of I/O interwoven with CPU processing (in the case of apps that aren't just CRUD over a DB).

I've seen a lot of dumb Rails server configs and even dumber usages of Rails without any tuning at all.

With a couple weeks or a month of work I could shrink the hosting fees or resources consumed by a factor of 5 out of most Rails apps that are sitting up around 30 servers... and probably a factor of 10. Leaving aside whatever rookie or even intermediate mistakes were made in their Ruby code or their database, this post indicates a lack of understanding of what happened when their server fell over. Proper tuning of a deployment should not trigger a 100% failure mode like this.

These folks were itching to get off of Ruby for whatever reason... after all their roots were in Java. If your goal is to do a rewrite and learn a new language and gain some notoriety why waste time learning what you did wrong with Ruby or your server config?

Your inability to understand their clear description of a basic cascading failure mode under load speaks poorly of your actual knowledge and experience. Given that, I have to take everything else that you say with a large helping of salt.

Their description of the root problem was very superficial:

> At some threshold above 50%, our Rails servers would spike up to 100% CPU usage and become unresponsive.

Yes, but why? What exactly were those processes doing? Why the sudden change at a particular threshold?

Their lack of detailed investigation into this makes their post useless to me -- I have no way of knowing (1) what specific aspect of Ruby's architecture makes it unfit for their problem?, or (2) is their application doing something stupid that causes the problem in first place?

TLDR: I'm butt-hurt that people are dumping Ruby. And because I hate Java I'll blame it on it also.

Not quite. He's saying that people who think Ruby is slow are often people who have no idea how to performance-tune it.

I'm getting the same experience here by entirely rewriting an old PHP web application [1] in Haskell using the Yesod web framework.

I've been using JMeter to benchmark both versions of the application. On a 10€/month dedicated server [2], the Haskell one was able to generate 220 dynamic pages per second [3] whereas the PHP one tops at 35 pages per second on a equivalent page.

Moreover, concurrence capabilities of Haskell are also pretty sweet : while I was benchmarking the web app using 2,000 concurrent connections, the application server was only using around 90MiB of RAM. I was not able to increase the number of concurrent connections as the client application I was using started to kill my quad core desktop, I suspect Haskell to be able to manage A LOT MORE concurrent connections as I didn't see any decrease in the throughput of the application as I was increasing the number of concurrent connections.

[1] http://files.getwebb.org

[2] http://www.kimsufi.com/fr/

[3] Static content has been ignored as modern servers like NGinx seem to be able to carry the static content (CSS, images, ...) at more than 5,000 req/sec on the same machine.

Interesting to hear of Go being used in production. It'd be great to hear some more details on your setup when deploying the go processes - how are you managing failover, what's your load balancer, and how are you handling swapping out processes etc? Are you compiling on the server or local machines before deployment? Most other languages have lots of solutions on the deployment side now but Go is so new there isn't much info out there.

Also, as you are running an API presumably your app in Rails was pretty simple, did you have the impression things would be more complex in Go if you were writing an app with an extensive front end and UI and using sql? I'd miss all the view helpers etc available at present I think. Going from 30 servers to 2 certainly sounds like a huge improvement, so it was definitely worth it for you, are you thinking of writing any front-end apps in Go?

I've been playing around with Go recently and it is a fantastic language for someone coming from dynamic languages like Ruby. I particularly liked interfaces as a way to define a contract for implementations to follow, and the simple package system which encourages you to make your code modular.

We actually have our own deployment tools for Go (and for Rails, our databases, etc). We built them before all the new hip options that are around today. We build on the target machines, although that's not a requirement since we all run the same architecture on our dev machines too (64 bit linux).

I'm not sure about writing a front end in Go, there's not a lot in terms of UI frameworks and Iron.io front end (HUD) is still in Rails so I can't really say much about it.

Interesting to know that you're building on target machines, I have yet to explore cross compilation, but it'd be great to be able to build before deploy.

Sorry, deployment was probably the wrong word - I was not so worried about getting the files up there (like you I have a simple home-grown solution for that), but more how you were handling process management, swapping out processes etc.

Perhaps with Go this is less of an issue because startup time for new processes is minimal, and you can simply kill one process, start another and not really miss any requests (unlike say Rails with startup times of 10 seconds or so for instances)? Did you find this wasn't a big issue in practice and something very simple works for you?

I'm currently playing with go but if considering it for client use would have to be sure that things like this were not an issue. Are you using any off the shelf tools for process management/load balancing or is it something that you have built yourself? Did you run into any problems?

For my (toy) apps, I've been compiling locally and pushing to the server. It's trivial to compile for a target platform and architecture, and you get a single compiled binary. Certainly my response isn't getting all the way to what you're looking for, but it's not difficult to write shell scripts that manage the deploy process from here.

Thanks, I use rsync wrapped in a script usually (because I'm dealing with more files than just a binary), haven't tried cross compiling yet but was going to give that a go. I suppose I was more concerned about what happens when things are on the server and how multiple processes can be managed. As soon as you have more than one and want to swap out server workers seamlessly and load balance etc it gets a little more complicated.

Ok, this bit has left me confused.

They were Java devs that liked ruby, they wrote applications in Ruby on Rails and the ruby apps were hitting limits so they immediately started looking at other languages.

But they don't mention the most obviously (to my mind) simple option.

JRuby

It is ruby (They like ruby). Most ruby apps can be run on JRuby with very very little changes (No need for a big rewrite) and it runs on the JVM with which they are familiar and it is very fast (and true multi-threading).

Maybe they did look into it but I would have thought that would be #1 on their "We tried this but discounted it" list.

In my experience, rewriting for JRuby is as much work as re-writing in a new language. Last I looked was a year or two ago, so it may have all changed, but it's not as simple as just moving your code over. Many really important gems don't work, or don't work properly.

That used to be the case, but a lot has happened in the last few years. Most of the time you'll be able to move a large Rails application to JRuby without any changes at all, except adding a Warbler config for deployment.

We use JRuby for a one of our backend applications in production, but we develop them with plain MRI Ruby. Only real problem we've run in to was some of our badly performing (badly written) parsing code that ran even worse under JRuby.

The only gems that do not work are those which are not pure ruby. For the popular gems I think there have been rewrites. I haven't looked into JRuby but it has changed quite a bit in the last year from what I read

I'd like to see some real benchmarks with a real app, but I doubt moving to jruby would bring the magnitude of improvement mentioned in the post.

They also did not mention why they didn't try any other of the more performant implementations of Ruby.

They probably needed a rewrite anyway, after learning more about the problem space they weren't happy with the api they'd made thus far.

Go wasn't the only thing that made their new stack better, but you can bet that Go would have made it very difficult for them to achieve performance as bad as ruby gave them.

That said, they are STILL using rails for the frontend stuff and Go for their more computationally intensive stuff IIRC.

They still kept some of their rails code, for things that were easier to keep in rails without hurting performance.

> "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go."

I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less.

That's how long it takes to go through the following materials (and fav some for later reference) and play with the language a bit.

http://golang.org/ref/spec

http://www.youtube.com/watch?v=ytEkHepK08c

http://commandcenter.blogspot.com.au/2012/06/less-is-exponen...

http://tour.golang.org/

http://golang.org/doc/install

http://www.youtube.com/watch?v=XCsL89YtqCs

http://golang.org/doc/code.html

http://golang.org/doc/effective_go.html

http://golang.org/pkg/ - use as reference

http://www.youtube.com/watch?v=f6kdp27TYZs

http://talks.golang.org/2012/splash.article

https://gobyexample.com/

http://golang-examples.tumblr.com/

And a some more similar things that you can mostly get to from golang.org site. The beauty of how concise the language and even its website are, is that you can literally just go through everything there one thing after another.

[1] This is my personal opinion based on playing with go the last few weeks/months. I'd love to verify this theory. It's not yet the primary language in which I do things in (I use C++11 atm), but for all my side tasks[2] it proved to be indispensable. And I found it very easy to pick up. I can't wait until I start doing all my work in Go, that will be a true test of its productivity efficiency.

[2] https://gist.github.com/shurcooL

It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal.

Any serious programmer should be a polyglot by default.

> Any serious programmer should be a polyglot by default.

It depends if you're going to spend years training someone or if you need an expert right now.

My experience is that it is impossible to maintain expert level skills in more than one or two language + library environments. You can remain familiar with other environments but you don't have the time to be an expert.

While I sometimes switch between C-family languages for different projects, it can take months to get up-to-speed with the changes in a language and its environment since you last programmed in it. I'm talking about situations where I know the language well but the environment has changed. Languages change a little but the libraries they use can change dramatically. And along with the change comes a whole body of implied knowledge about how to safely and effectively use it all and this impacts on how you use the language itself.

If you've literally never programmed in a language before, it can be a few years before you know about all the eccentricities, before you understand why the language follows certain patterns, before you understand the risks with certain behaviors.

If you need an expert now, not in 12 months time, then they need to know both the language and the environment. If you can wait a couple months, then they still need to know the language.

"It depends if you're going to spend years training someone or if you need an expert right now."

I'm not going to claim that there are no legitimate reasons to hire people who are narrow experts in Blub (and only Blub), but I'm having a hard time thinking of any.

At an established company, you've got the luxury of time -- there's rarely a good reason to "need an expert right now" that isn't just a contractor. At a startup, where hiring the wrong person is a disaster, hiring an "expert right now" is like holding a loaded gun to your head. Ideally, you should be hiring "T" people -- lots of breadth, with lots of depth in at least one area.

A truly good programmer will pick up your language/framework in days, and be at full productivity in months, even from scratch. It's hard to do better than that.

Even at an established company, a really smart language newbie could write a bunch of ad-hoc code that does almost the same thing as well-understood libraries that everyone more experienced in the language uses. Even if his code was relatively high-quality, now you have a bunch of extra stuff to maintain, and everyone who interacts with it will have to learn this thing instead of just using the library everyone knows. I've done this in languages I'm unfamiliar with and will probably continue to do so.

Not Invented Here syndrome goes away with general experience. It doesn't come back every time you switch to a new language.

Just because I've never written Erlang doesn't mean that I will automatically try to write a random-number generator (say) the first time I need one in Erlang. I have enough experience to look for a library function first.

Empirically, NIH tends to be more common in single-language developers, not less. People who place a lot of value on their "expertise" in Blub tend to do so because they're over-weighting the importance of their memory of the API details. When they don't automatically remember something, they leap to the conclusion that it doesn't exist. They're also typically a less-experienced cohort than people who have written in lots of different languages.

I wasn't talking about NIH syndrome, I was talking about "I don't know that a common library exists for this standard use-case so I'm going to write my own one-off because I have a job to do". I mean, you can google for libraries but sometimes you just don't find them and then find out a few weeks later what you should have used.

You talk as if we develop code by chiseling it out in stone. We write it in text files, usually with IDEs. No one needs to learn the new libraries the newbie made, they can talk to the newbie, point out what the library already gives (and where to look in future) and write to well known interfaces.

Any project should have time allocated for "tech debt" and that's where this sort of thing gets addressed.

"I'm not going to claim that there are no legitimate reasons to hire people who are narrow experts in Blub (and only Blub), but I'm having a hard time thinking of any."

Blub is a very useful tool but has a bunch of quite esoteric gotcha's that an expert will be able to avoid due to long experience.

Expert has used almost all of the Blub framework over the years, including the more popular third-party addons and already knows what's best and fastest in a variety of situations without having to think about it much.

"A truly good programmer will pick up your language/framework in days, and be at full productivity in months, even from scratch. It's hard to do better than that."

Yeah, but then HR won't be able to tick off the 'Blub expert' box and, well, nobody would ever get hired! Or something...

> impossible to maintain expert level skills in more than one or two language + library environments

And the crappier your language and libraries, the more time it takes to be an "expert."

By this definition, Ruby/Rails are pretty crappy (I kind of disagree). As a newcomer, the amount of stuff going on in a Rails app and the stack trace when there's an error are pretty overwhelming. Meanwhile, people expound on how simple and elegant Rails is. Lately I've been thinking that this is because those people started using it 5 years ago when it was small and their knowledge has built incrementally with the environment.

My other theory is that it's a lot harder to track down bugs for a newcomer because like C (and unlike Python in most cases), importing functionality from another file is implicit. That is, when you 'require' a bunch of files, there's no indication which functions are coming from where. For me, this is one of the things Python solves marvelously (it's generally considered bad form to 'import *').

Rails is pretty crappy if you ask me. It's "omakase" which is Japanese for "acts according to what DHH wants despite what the community wants". And there's a lot of magic happening that isn't explained very well. There are better frameworks in Ruby. Ruby itself doesn't take all that long to be an expert at.

>Rails is pretty crappy if you ask me. It's "omakase" which is Japanese for "acts according to what DHH wants despite what the community wants".

This sounds great. I'd hate software made in the way some "community" wants. Community is the other name for committee.

Though I could argue that all the languages pretty much take the same amount of time to become an expert. Just because some languages are supposedly higher-level than others doesn't mean that the complexity they allow you to tackle (and associated challenges as a programmer or "expert") is any less.

Especially since being a good programmer is more about design, choice of interfaces, reactivity to change, etc. which are by definition language agnostic.

I firmly disagree that Ruby requires any less effort than say C, C++, Java or LISP to become an expert.

Ruby doesn't take long to be productive in. It is an utter pain to become expert in, for any reasonable definition of "expert". Fortunately it's fun enough that I don't mind the slog.

"it's a lot harder to track down bugs for a newcomer"

This is FUD. It's a different way of doing things, not harder. I live inside Pry which makes it rather easy to figure out what's going on.

That's kind of what I'm talking about. I've never heard of Pry. With Rails I have to learn about the 100 most common gems (devise, paperclip, mongomapper or mongoid), Pry (thanks for that), bundler, rvm, and ActionEverything before I can be productive (or understand a simple app) . With Node.js, something newer with less "maturity", I figure out npm and I'm good to go.

I really don't think it's FUD to say that Rails has gotten much bigger in the past 5 years, and it's definitely not FUD to say that as codebases and tooling grows, so does barrier to entry.

The specific thing I guess you're objecting to is that it's harder for a noob to understand implicit imports and where something is coming from if you don't know much about what you're importing. If you use a tool to solve that language deficiency, that doesn't remove the deficiency from the language. By that logic, adding an IDE to Java makes it a very concise language.

However, you do not need to be an expert to be productive. You can write Django web apps perfectly well without knowing about Python's metaclass tricks, for example.

An expert still needs time to become familiar with a large codebase and architecture. I wonder if a competent programmer could simply learn the language in addition during this period.

Until your form isn't coming out quite right, and you need to start digging around in the source for the form class to figure out why. The happy path generally doesn't need expert-level skills, but debugging often results in a quick deep plunge.

Go as of today doesn't have too many libraries available. So there is really nothing to learn besides the language itself.

That's not really true.

Granted Go doesn't have as many libraries as the more established languages, but to say there aren't many and there's "nothing to learn besides the language" is just flat out wrong.

Just for starters there's http://code.google.com/p/go-wiki/wiki/Projects

A seasoned Java developer is expected to have worked with one of the mainstream ORM like Hibernate or Ebean. But a Go developer gets away with not having to know an ORM because there is no mainstream ORM in Go. :)

So what, the lack of a mainstream ORM implies "So there is really nothing to learn besides the language itself"? I don't think so.

And there's probably more to a lack of an ORM other than "Go is immature." It's a fairly common opinion among the Go community that ORMs are not worth their complexity. I tend to share that opinion myself, after having worked with a few in a couple different languages.

ORMs are broadly considered antipatterns in Go.

A seasoned java developer stays away from orm anyway ;)

What, you don't know about The Programmer Hierarchy?

https://news.ycombinator.com/item?id=1622553

http://lukewelling.com/wp-content/uploads/2006/08/programmer...

I find a lot of people who use high-level languages are terrified of tediously direct contact with the machine, and a lot of people who use low-level languages are terrified of the performance costs of abstraction. I'm terrified of both.

I think that in general, C++ is a lot harder to get your head around than C, being for all intents and purposes a superset, and an easily five times bigger one at that.

Love this comment, "I'm terrified of both."

That's how you gauge the experience of a programmer: how much of the field she/he's terrified by.

EDIT: I originally meant this as a joke, but seriously, if someone accurately knows where the gotchas are, that's valuable. Also note if they're biased to false positives and/or false negatives, and by how much. Are their heuristics for dealing with unknown territory efficient and likely to converge on good approximate results?

> That's how you gauge the experience of a programmer: how much of the field she/he's terrified by.

I'm terrified by what I don't know, especially by what I don't know that I think I should know, to be "in the know".

Thanks guys, I guess I really have had some rough times. I like how Lisp and Assembler at the top of the hierarchy capture the two extremes. Some hypothetical language in the middle would be great, but maybe the best we can do is to straddle that point, e.g. with C++ and Python.

Rather than straddle the middle, I think I'd go with as high-level a language as I could get, coupled with a simple low-enough-level language to get whatever performance benefits I needed. Some combo like Python/C, Clojure/Java, or maybe some other lisp dialect and C.

> I like how Lisp and Assembler at the top of the hierarchy capture the two extremes.

Not really. I'd place Agda or Coq above Lisp.

C++ and Python pretty much run everything, everywhere. Standard, open languages are hard to beat when you want to fully control your development stack and not worry about future control issues. I wish go was an ISO standard like C++, I'd be more interested if it was.

Fantastic! I'm primarily a C# programmer and have first-hand experience of this being true.

I agree with the premise that, as a professional software engineer, it is my responsibility to be a polyglot.

As for myself, when I started writing Python, I mainly wrote C or Java code in Python - in a similar way, perhaps, how early C programs were littered with __asm__() constructs.

It took a long time to learn how to write things in (though I hate the term) a "pythonic" way. That is, to learn the common language idioms that are not taught in any tutorial, or are part of pep8, which illustrate common patterns in Python code, the ins-and-outs of PYTHONPATH, etc etc.

So while I agree that, in a weekend, a reasonably proficient programmer can pick up reasonable proficiency in a given language, being a "Python Programmer" to me means that one has developed an intuition for the common patterns, libraries, pitfalls, platforms, and clever specific features of a given language and its ecosystem.

I agree, but I think the age of the language and community is another factor. Java, Python, C++, these are all old languages with decades of history and habits. Newer things like Node.js and Go have no history, no baggage to learn or avoid. I think starting in something with such a clean slate is somewhat easier because there is less ecosystem to learn.

I once heard a quote like this "programming languages are frozen knowledge about software engineering". (Anybody got a clue who said something like this?)

New languages usually improve upon older languages by making certain errors impossible by design. For example, Go allows pointers, but not pointer arithmetic. With C we learned pointer arithmetic is often harmful (though sometimes necessary), so Java removed pointers from the language (not completely, though). Go takes a sensible middle way, because no pointers sometimes is ugly. Such a clean slate is good, because with older languages programmers fight their language deficiencies with habits (e.g. if (5 == x) instead of if (x == 5) to prevent if (x = 5)).

On the other hand, we will find the pitfalls and dark corners of Go over time, but currently we do not know them enough. Go will acquire its own conventions, but we do not know all of them yet. This is the risk of a clean slate.

> On the other hand, we will find the pitfalls and dark corners of Go over time, but currently we do not know them enough. Go will acquire its own conventions, but we do not know all of them yet. This is the risk of a clean slate.

That's bullshit, Go is not a "clean slate", and a number of design issues of Go are known and have been known from the first release regardless of its designer's refusal to acknowledge them. We know pervasive nullable types are a source of errors, we know shared-memory concurrency is an error-prone default, we know a lack of generics makes userland code painful and generics are hard to retrofit in an established language (and even Go's designers know it, why do you think they build special-case generic collections in the interpreter?), we know allowing implicitly ignoring errors is a bad idea and making it easier to ignore than handle errors also is. These are not recent issues, they're well known and there are a number of possible strategies for handling them.

And Go's worst sin, to me: we know that foisting complexity and repetitiveness upon the user leads to forgetting, and forgetting leads to mistakes. And that's exactly Go's approach to errors, resources management and shared structures mutability. Human error is something you can very reliably bet on, human infallibility... not so much.

Funny you should take Go as an example, since it's often bashed for not having taken the last decade of language design into account.

Go does take the last decade of language design by Pike, Thompson, and Griesemer into account. Mostly Pike as I am not sure if the other two guys even did design some language in this time.

Personally, I consider gofmt the biggest achievement of Go, if it manages to make that mainstream. While there are equivalent tools for C they are not widely used.

> Go does take the last decade of language design by Pike, Thompson, and Griesemer into account.

Decade which ended 20-odd years ago relative to the rest of the world, kind-of the point.

There's a difference between not taking it into account, and not believing it is worth including.

This has been my experience learning Go as well.

This is why Joel Spolsky correctly observed that the replacement of C/C++ and functional languages with Java in university CS curricula is a tragedy. If you don't understand pointers or recursion, you are not a polyglot and you cannot pick up just any language in a matter of weeks.

The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer talent.

My intro programming course (in high school) was in C++, and I think it would've scared me off programming entirely if I hadn't picked up mIRC-script (a very different language) on my own as a hobby. There is so much accidental complexity in C++ that it's a pretty terrible introductory language. We literally spent several weeks of the semester on how to get input and output to work properly through the giant mess that is iostreams.

At the introductory level, I think the biggest issue is teaching "computational thinking" [1] or "procedural literacy" [2]: getting people thinking about the idea that you're writing specifications for a machine to carry out computations. From that perspective, it's best to pick a language that lets you get to algorithmic logic as quickly as possible.

[1] http://www.cs.cmu.edu/afs/cs/usr/wing/www/publications/Wing0...

[2] http://dm.lcc.gatech.edu/~mateas/publications/MateasOTH2005....

Me too (well, ok, my first programming class was in Fortran taught by a 85 year old man who spent most of the time telling us about how much harder it was back when you had to use fortran).

I hated C++. I still think it's a fairly terrible language. But, for fun I took the Harvard CS50 course to refresh my knowledge of C and I found that WAY better than my C++ course. I think C is brilliant for introducing programming because it's very very simple, yet also very very difficult. There's not much to learn, except a lot of concepts (memory usage, data structures, etc).

I also think Objective-C is a really great language though, so I might be crazy. But, you give me a choice between C++ and a language that is basically C with a few additional keywords and garbage collection, and I find that an easy choice...

C++ is terrifying, and teaching an introductory programming course in C++ is an awful idea. Using plain C would make much more sense. C is also a better choice than Java because Java doesn't force you to learn about pointers, or memory in general, really. Countless Java bugs are introduced by programmers who don't understand what operations give you a copy of something, and which ones give you a reference to something (especially if that something is mutable).

I also think Objective-C is pretty good though, especially with the addition of ARC. It's like a better C, with Smalltalk-style objects and a lot more (mutable and immutable) datatype options. Much saner and less painful than C++, but more challenging than Java.

Java doesn't force you to learn about pointers, or memory in general, really.

Countless Java bugs are introduced by programmers who don't understand what operations give you a copy of something, and which ones give you a reference to something

These two statements seem to be at odds with each other.

The Objective-C garbage collection is deprecated.

The problem complicates further with the fact that IDE fever grips you very early on while using a language like Java.

When I started with eclipse + Java, within hours my perception of Java was that it was really more of 'fill up the blanks' rather than programming.

And I think that is a valuable lesson too. As a developer, the end-goal is to produce something of value, working software, not spend hours frowning over pointers and references. That was my early experience with C/C++, in any case. I did end up finishing my assignments (ranging from a fractal generator via a microcontroller operating an RC boat to a simple 3D FPS game), but I did feel a lot of overhead with non-functional things.

Of course, you get the same thing in Java when you first encounter a NullPointerException, ;)

This became clear to me when suggesting using a language other than that supported by Visual Studio to one of my previous teams. Their reply of, "Can't do it; no IntelliSense" shocked me completely.

I've seen C/C++ programmers failed in writing good readable code.

C/C++ programmers tend to do premature optimization (habit) due to the culture and the problem domain (past experience): device driver, kernel code, game development.

How important Pointer is for say learning any high-level programming language that don't have pointers (pretty much everything outside C/C++/Objective-C)?

Sometimes I felt that knowing Pointer arithmetic and tricks are some sort of chess-thumping that old programmers tend to do. (My background is system programming so I know how pointer works).

It doesn't matter what you know or learn during the university. What matters is whether you continue to hone your skill or not and have a high bar of quality.

I think he was talking about pointer from the "idea" point of view, not the language feature.

References in high level languages are mostly hidden pointers. A good knowledge of how pointers work is really important to get a good understanding of how high level languages work (a lot of Java programmers don't well understand the distinction between value and reference types, for instance).

Likely, be at ease with recursion is also very important as many complex problems are recursive by nature, and so way easier to solve using functional techniques.

Yes. Pointers are always there in programming and you need to understand how they work, even if you are using a language where you don't have to declare them as such explicitly.

https://en.wikipedia.org/wiki/Hasty_generalization

I feel like this article on generalization could be a biography of my life based solely on the introductory paragraph, but I don't have enough time to finish reading it to be certain.

"Sometimes I felt that knowing Pointer arithmetic and tricks are some sort of chess-thumping that old programmers tend to do. (My background is system programming so I know how pointer works)."

Or the best, easiest and clearest way to do things... Sure you can write confusing crap with pointers, but they are very useful.

> The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer talent.

From what I can experience in the multi-site enterprise projects I participate, the industry is being quite successful at that.

Successful at commoditizing programmer talent? Perhaps.

Successful at producing high-quality software, relative to the resources they are expending on it? Not so much.

> Successful at commoditizing programmer talent? Perhaps.

Yes this is what I mean.

In most enterprise companies nowadays, developers are seen as easy to replace programming cogs.

> Successful at producing high-quality software, relative to the resources they are expending on it? Not so much.

Who cares about quality when the price is right?! :)

Being sarcastic here, I am the opinion the software industry should be under the same quality regulations and expectations as the other industries.

Usually most people return stuff that does not work properly, while with software they just live with whatever bugs the software has.

I agree that any serious programmer should be able to be productive in any language, but mastery of a style of programming takes time.

It's easy to move about in the same style of programming, but trying to move someone from enterprise-y OO (Java, C#) to functional (Haskell, Lisp) or even Go is a bit of a leap. Concepts often just don't cross over. A Go channel, for example, makes sense to an Erlang developer, but not a Java developer. Goroutines don't make sense to someone familiar with traditional threads.

There's a certain amount of rewiring that needs to be done. Looking for a "[language] programmer" is strange, but looking for someone proficient in the style of programming used makes sense to me.

Even within languages, styles are not set in stone; e.g. are you the OO-style Scala programmer, or are you the FP-style Scala programmer?

Java has had channels for a while now at the library level, they just lack pretty syntax. But this just reinforces my point.

> It's quite strange to me that people would identify as or look for a "[language] programmer".

I understand people doing this.

First you can be more sure of what you're getting. It's sad that tests like FizzBuzz are so useful, but it's a fact. If I hire a Java developer for a Java position, I can figure out there Java skills. If I look at a PHP developer, it's more of a crapshoot. They may have a lot of PHP experience, but can they convert that to Java or have they just been doing cargo-cult stuff? This would be a smaller issue with something like C# that's closer to Java.

Second, and I suspect more common, is time investment. It might take someone quite a bit of time to switch languages. If they haven't done it before, you might find out it's a weakness for them. In my experience, at least in smaller companies, the fact that you're hiring means you need someone now. That extra time could be killer, because you waited to long to start hiring.

I would at least expect candidates to look into what we're using. When I changed jobs 2 years ago, one place I applied was a Python shop. I've never used Python professionally, but I've tinkered with it. If you're applying for a Java position, I expect you to have at least looked at Java before the interview. Sadly, I bet that would happen a non-trivial number of times.

How did you convince them to hire you when you had no professional experience? Did you have to take a pay cut to do it?

I didn't get hired for the job, after the first interview I bowed out. I ended up in another mostly Java position (although we've also started doing some Obj-C).

I had 4+ years of working on a reasonably sized and complicated Java application. In addition, a few years before I used Python for personal projects so I could show some experience.

But it turned out the position was largely for front end development, and not some of the more back-end stuff I'm interested in. I think they liked me (don't know how I compared to other candidates) but I thanked them for their time and told them the position wouldn't be a fit for me.

If your code in that language is good enough then you can usually get a job for it. Of course, I'm talking about places where programmer/HR department have a big overlap. I don't much like working for the other places.

If you're looking for a good programmer, it often pays to be looking for one in a niche language. Paul Graham referred to this as the Python paradox: http://www.paulgraham.com/pypar.html

(The inverse of this is if you're looking for a PHP programmer, and posting ads on monster.com ;) )

Try a Lisp. You'll be humbled. In my opinion, anything resembling C is easy stuff. Pointers, threads ... it's all pretty easy, really. Truly breaking out of that C mold is the differentiator though, in my opinion. Anybody can learn C, Python or Javascript.

Lisp is a huge paradigm shift, but once you get that writing code in Lisp is basically just writing parse trees directly, it's not so scary.

If you don't already get recursion, though, you will struggle. Everyone struggles with recursion at first.

I completely agree.

I think to completely explain this phenomena, one would have to make references to people's skill levels, and it's hard to explain it in a few words. So I'll just not say anything at all on this subject beyond that.

If only the morons that hire could figure this out. I almost NEVER use Java, but obviously I have and CAN use Java. Yet interview after interview demands that I use Java day to day in my current job to be considered. And this is for data transformation/analysis jobs, in which my use of Python makes me dramatically faster and more productive than my Java using colleagues.

Of course use of the word "moron" might be poisoning the well a bit. "Possessing incomplete information" might be a better way to put it.

Generally I don't understand "C++ programmer" to mean a programmer who is an expert in C++, and C++ only, but rather a programmer who may be comfortable in many languages but has a special expertise in C++. All my team identifies as C++ programmers, but we all speak Python and Java, and some can do Haskell, etc.

It takes a long time to become really good at C++.

"Any serious programmer should be a polyglot by default."

The language itself is not usually a problem for me. I mean an array is an array in any language. It's environment. Example: c# (which I think in MS's best work, by the way); c# was easy to pick up. The .Net framework was another matter. That took time. Knowing where the resources are and hell - even getting comfortable with the documentation and the IDE took time.

So now, when I hire a programer, I'm not too interested in his language skills. I'm assume he can code well enough not to embarrass himself. It's his knowledge of the environment that we run. That's what I look for.

>Any serious programmer should be a polyglot by default.

My own experience is that there are PLENTY of programmers who have used, say, PHP and Python, but who you wouldn't want to touch your C codebase in a pair-programming session.

If you've used C++ and C, then sure, you can probably jump to just about any language. Someone who's only used Python (or worse, Java or PHP) will likely be a danger to themselves and others for the first year or three using C.

Starting out with Python as a newbie is disastrous.

I started out with assembly, then went to C and then all the C-family based languages. Non C-family languages are actually rare enough to neglect for 99% of all programming tasks today.

Always remember going from something difficult to simple is easy, but going from something simple to difficult is not easy.

I disagree. My path went like this:

HTML->CSS->php->javascript->C#->Visual Basic->Java->Ruby->Python->C->Go->Scheme->Lisp->Haskell

I believe the "python way" and "pythonic" code that the community pushes is great for newbies, since it gets them thinking about not just writing code, but writing it idiomatically.

This, in turn, makes most think about why doing it the "pythonic" way works better.

>Always remember going from something difficult to simple is easy, but going from something simple to difficult is not easy.

Great. There are plenty of hard working people to make this "piece of wisdom" completely worthless.

>Starting out with Python as a newbie is disastrous.

... why?

>My own experience is that there are PLENTY of programmers who have used, say, PHP and Python, but who you wouldn't want to touch your C codebase in a pair-programming session.

There are plenty of programmers that wouldn't want to touch C to begin with, regardless of past experience. Anyway, why would you ask somebody with no experience in C to develop your C codebase?

>If you've used C++ and C, then sure, you can probably jump to just about any language.

Nope.

>why would you ask somebody with no experience in C to develop your C codebase?

The message I replied to said:

>It's quite strange to me that people would identify as or look for a "[language] programmer".

So what I was saying was that you would want a C programmer to develop in a C codebase.

>There are plenty of programmers that wouldn't want to touch C to begin with, regardless of past experience.

I wouldn't want to do a lot of original development in C myself. I have used C -- and assembly language -- to do nontrivial things in the past. But I'm ready to be done with C myself.

>>If you've used C++ and C, then sure, you can probably jump to just about any language.

>Nope.

Umm...[citation needed].

I doubt that. Learn a language's syntax and semantics, perhaps, if it's not C++. Learn its standard library so one uses it aptly rather than re-inventing, less likely, depending on the size of that library. Become proficient in the language's idioms, know its gotchas and what are the more efficient of the choices it presents, less likely still.

Code in it, yes. And I agree with the polyglot.

>>Any serious programmer should be a polyglot by default.

The quality of a serious programmer is to get things done. Not tool tricks.

When I go out to hire a carpenter, I just ask him to show me some of his work. If I like it I buy from him or hire his services. I don't go around and ask him to demonstrate how he uses a hammer.

Getting things done means using the best tool for the job. Sometimes that's a language outside of your immediate preference, because that's what the best tool/library/platform is written in.

Today I wrote Java (LDAP server plugin), and ssembly (for unit testing some code that has to minimally interpret x86 assembly). I'll also occasionally need to write some JS/HTML/CSS for the web, work on systems programming for other architectures (ARM, AVR), write kernel code and drivers (C/C++), write/extend a Python script (HTTP test client), write Tcl (to drive simulavr), ObjC (iOS apps), and occasionally write some functional code for fun (I have no business case for it, sometimes it's just nice to work on something clean).

I've done and do all those things (and more) not because I have 'tool tricks', but because in nearly every case, I needed to use the tool most suited to getting things done in that problem domain.

Programmers that refuse to adapt to the given problem domain do their users a disservice. It's like the old joke about which nationalities run heaven and hell -- the worst possible people set to tasks for which they're innately ill-suited. However, programmers can learn more -- if they're willing -- and adapt to better suit themselves to the problem at hand.

He doesn't use a hammer though, he prefers to use his screwdriver for everything. What you didn't see from the pictures of his work is that they have now burnt to the ground.

Burnt to the ground on purpose because there were too many bugs or it became an impossible to maintain tangled mess of meaningless symbols.

By the way, since he only uses his screwdriver and you project requires nails, he'll take 5 times longer than your deadline.

Your end product will work for about 2 months, then you'll pour the gasoline on it and burn it down yourself.

Absolutely. Even if you're a blub grandmaster, you really ought to know a dozen other things fairly well.

>> Any serious programmer should be a polyglot by default.

I can pick up a new language very fast but won't consider myself a polyglot (know a handful of languages). I do still consider myself a serious programmer.

> Any serious programmer should be a polyglot by default.

Fully agree.

At the end of my CS degree, I was able to code in:

- Pascal

- C

- C++

- Prolog

- Smalltalk

- Camllight

- Java

- Assembly (x86 and MIPS)

- PL/SQL

Many programmers are religious about their languages.

> I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less.

Agreed! My career is mostly database development. Very high level. Yet I felt comfortable in Go in a few weeks. In a few months I was handling low-level stuff (to me, anyway) like tweaking Go's web server. Go's official online documentation is excellent, although I wish it had lots more examples. The next best learning resource I have on it is the book Programming in Go (also get the Go Programming Language Phrasebook).

Re [1]: I think that's a reasonable statement. My issue with it, however, is that after just a few weeks (or even a few months), said developer might still not know about various design pitfalls or best practices in the new language.

In short, they might not be writing idiomatic code, and you'll end up maintaining it for a long time. IMHO, becoming basically proficient with a language is very different to being able to create software which will scale well to large teams, while also aging gracefully over time.

In general I agree, but I think it's important to note that while a lot of language skills are interchangeable, going from an objected oriented language to a functional one is a pretty big step.

It took me 1 week each to go from 0 to being productive in javascript & ruby and it took roughly the same time for some other developers in my team. I think it's more about knowing where to start, knowing various paradigms, etc than the language itself.

Thanks for the links man.

Yea, michaelochurch have talked about that one before.

wow. these are very good... thanks.

Excellent success story with Go.

Love this quote... "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go."

That's really obvious but overlooked. Developers just looking for their paycheck will only have learnt the established languages.

Developers who enjoy their profession will look to the future, excitement gleaming in their eyes.

I've heard of business developing in Haskell a good amount of success. I suspect, for the same reason.

It could also be said that developers who use established languages are excited to get things done rather than reinvent wheels continuously.

the lisp community used to hold this as a truism - there are some languages so good (powerful, satisfying and pleasurable to work with) that they become a compelling reason to take a job regardless of the nature of the actual product.

Yaron Minsky of Jane Street said pretty much the same thing about OCaml in one of his talks. And it makes sense. Go and OCaml are not mainstream, so the people who know these languages are likely interested in improving their own skillset, and it makes sense that they would be above average programmers. If they like the language enough, they would of course want to work with it rather than other languages.

“the entire process started up with only a few hundred KB's of memory (on startup)”

There is no way this could be true: last time I checked, a bare minimal Go HTTP server requires at least 2.8MB of memory on 64-bit machine. Are you using the default net/http library?

Yeah, my HTTP server made in Go usually starts up at around 6MB or so.

This story sounds mighty familiar. I realize the topic is about one language over another, and most of the comments here seem to be focusing on that, but there's another part which I find fascinating: the cascading failure mode when they'd lose a web head.

"This would in turn cause the load balancer to think it failed and take it out of the pool, thereby applying the load that the unresponsive server would have been handling to the remaining servers. And since the remaining servers are now handling the load of the lost server plus the spike, inevitably a second server would go down, the load balancer would take it out of the pool and so on."

It was 8 years ago, and the customer was running PHP, not Ruby, but otherwise it's the same basic story that lead to the creation of "Surge Protector", or its actual name, "Suicide Pact". http://rachelbythebay.com/w/2011/06/28/sp/

Keeping in theme with the suicide pact, STONITH, the other kind of pact for high-availability machines comes to mind - Shoot The Other Node In The Head.

Why did go "win" over erlang, scala, and clojure? Because it's becoming trendy and "fun?"

And, why are "fun" and "joy" terms ruby bloggers use for languages? Is this code for "easy" and "familiar?" As in, "this language is easy to learn.. It does not require us to learn difficult but mind-expanding concepts to become proficient."

Replacing ruby/rails with something (just about anything!) results in far fewer servers. Isn't this obvious to all? Is this really news to the average HN reader?

If that's an I/O bound API server, bet those two servers could be brought to one in a language/runtime that's more productive but a little less "fun" to learn.

One could adopt some kind of event-based system in scala or clojure, but Erlang and Go are alike in being the lone runtimes where running millions and millions of very small messaging processes is AOK no problem for the runtime, and not something one has to work really hard for.

Fun and joy are terms applicable here because the alternative is Erlang. Zing!

Go is rather ideal for these guys use case: if it wasn't an easy win, given what a snug fit their use case is, there would've had to have been red flags abound, as this really is a nearly idealized work load for Go's use: a hell of a lot of processes which sit around doing nothing, where one occasionally gets a message and forwards it along. Perfect Go story, as Go's lightweight processes (goroutines) are ideal for this kind of Communicating Sequential Processes routing workload.

>One could adopt some kind of event-based system in scala or clojure, but Erlang and Go are alike in being the lone runtimes where running millions and millions of very small messaging processes is AOK no problem for the runtime

Is that really the case? Scala's Akka actor library seems like it would qualify just fine: http://letitcrash.com/post/20397701710/50-million-messages-p...

Haskell's runtime makes for cheaper threads than Erlang. Possibly cheaper than Go's, too.

Fun and joy doesn't always mean easy and familiar. For instance: Haskell is fun, but it isn't always easy or familiar.

> Why did go "win" over erlang, scala, and clojure?

Erlang sucks at string handling which is a no-go in Web developing.

Scala, Clojure, people don't even bother to setup JVM. Python is popular in some degree because it's installed by default on Linux and OS X.

This could easily be rephrased as "How We Went from 30 Servers to 2: Static Typing and Compiled Code".

I rewrote my DNS checking tool (http://www.dnsinspect.com/) in Go and I saw huge differences in resource usage (previously it was implemented with Ruby on Rails + EventMachine), my memory usage went down from 128MB per background worker to a few KB.

Now I'm able to run hundreds to thousands of concurrent reports using a small VPS, the Go application is using 36MB of RAM (24MB front end + 12MB for background workers). Go language is well suited for my particular case (many concurrent IO operations).

This was my first Go project, in a week I was comfortable with it, I had many alternatives but I really liked the simplicity of Go, fast compilation and easy of deployment. Because I've missed some pieces from the Ruby's world I've combined Jekyll (Compass, HAML SASS, RedCarpet, etc) with Go. :)

Looks great, bookmarked.

Do you also used the default net/http support libraries?

Yes, I'm using net/http.

I'm missing the connection between poor API performance under Rails and the decision to do a ground-up rewrite. Was the initial performance problem due to an API problem or a platform problem? Was there any profiling of the poorly performing API server? I'd love to hear the story behind that analysis and how it impacted the tale told in TFA.

I ask all this since I've lost count of how many times I've participated in or witnessed an averted rewrite via a good dose of profiling and a few key bugfixes. I'll acknowledge that's not as fun as a clean-slate project, but vast amounts of engineering time were saved.

I agree with the 'Go is fun' (and easy to pick up) sentiment. I spent a few weeks recently writing an NTLM library for Go for a consulting client (I hope they'll let us open source it at some point). All the parsing and bit manipulation was extremely straightforward due to the excellent standard libraries and I was able to quickly move from not knowing Go, to a fully implemented NTLMv1/v2 library!

Another day, another Go PR post on HN.

What is new about porting code running in one of the slowest interpreters around to a compiled implementation of another language?

I was already porting Perl/TCL code to C++ back in 2002 with similar performance results.

Don't kids nowadays learn anything about performance in their CS degrees?

It's an interesting tale, and I honestly have no bias toward Ruby or Go, but the cause/effect relationship is poorly illustrated (we switched languages). We all know runtime / performance / scalability is more complex than that. I'd like to hear where the bottleneck(s) were and how Go solved them.

Last month I picked NodeJS and build couple of sites with it. This month, I want to pick up something new. I was hesitating between Python and Go. Can you answer couple of questions about Go from someone coming from NodeJS:

1- What is the state of the external Go libraries, especially DB (MySql), caching libraries (memcached), protocol libraries (Oauth). Are they stable 100%

2- How easy is logging and tracing in go?

Thanks.

Memcached's author (Brad Fitzpatrick) is a Go user, and wrote a memcache client for Go.

Go has a bunch of good stuff surrounding SQL. I haven't used MySQL, personally, so I can't comment on which MySQL driver is best, but I know YouTube uses Go for something related to MySQL, though I can't say for certain exactly what. http://code.google.com/p/vitess

The goauth2 (http://code.google.com/p/goauth2) library is written by Brad Fitzpatrick and Andrew Gerrand (two members of the Go team), and is stable.

Logging is great. Interfaces make it really, really flexible. I'm not sure what you mean by tracing. Stacktraces? Those are easy to retrieve: http://golang.org/pkg/runtime/debug/#Stack

Well, Go has a bunch of libraries for different SQL servers. I wouldn't go so far as to say it has lots of good SQL stuff; it's in approximately the same place as C is w.r.t. databases.

Having to interface with complicated SQL is a reason not to use Go (but not an insurmountable one).

Brad Fitzpatrick is not just a Go user, he is a core Go developer :-).

There's also the mgo library for MongoDB which is probably one of the best Mongo libraries out there (even 10gen thinks so): http://labix.org/mgo

And http://github.com/lib/pq for Postgres.

I'm the author of memcache client and server in go [1]. The client is much faster than bradfitz's client when serving hundreds of simultaneous workers - try comparing them with go-memcached-bench [2]. There is also memcache server [3] written in go, which can cache much more data comparing to the original memcached thanks to ybc library [4].

[1] memcache client and server library https://github.com/valyala/ybc/tree/master/libs/go/memcache [2] go-memcached-bench https://github.com/valyala/ybc/tree/master/apps/go/memcached... [3] go-memcached https://github.com/valyala/ybc/tree/master/apps/go/memcached [4] ybc library https://github.com/valyala/ybc

Python and Go are my languages of choice. I have found both share a the philosophy of less typing is more and more standard library is more. For web apps I have found Python has everything I can think of and more as far as third party libraries. Go package management and deployment once I got it I wondered how I could ever live without it. 1 - SQL is supported by Go specifically MySQL has a couple of drivers to choose from. After using sqlalchemy [1] everything else I have ever used just seemed like the dark ages. 2 - Go logging is standard and has been a couple third party options depending on your tastes. Tracing I think you mean stack is pretty amazing when my app crashes it is as easy to debug as any VM (JVM, Python VM or Ruby VM).

Python gives me a lot of choices based on the type of application I am building monolithic (DJango) or light (Bottle) and many package management and environment setup tasks (as with Rails, Java nightmares ect). Go on the other hand takes care of all the package management and environment setup almost non existent. YMMV good luck. [1] http://www.sqlalchemy.org/

Can someone tell me why they would use memcached over redis? Isn't redis a superset of memcached? I don't see a use case for memcached anywhere in a stack any more.

Has anyone had an experience of running websites from Go; or more specifically, how you handle none-HTML content?

I've been considering porting my CMS from mod_perl to Go, but I'm not sure how you work with the other files (CSS et al). I did read somewhere that you run Go from Apache but it's not recommended.

I've just started creating toy projects in Go. I put these Go projects (and Apache, which runs older PHP projects) behind nginx.

For hosting CSS, you either generate it programmatically and send the right Header, or you can use http.FileServe from the standard library [1]. (Surely other approaches are possible, but those are the two I've played with so far.)

[1] https://code.google.com/p/go-wiki/wiki/HttpStaticFiles

Thanks for that.

The issue I have is that I would prefer to keep Apache around if I can (I have a few sites - some of which I host for friends). While my web server does have a few IPs attached, I'd rather not have to buy more IPs just to separate Apache from Go.

And to be perfectly honest, I do quite like Apache. (each to their own I know, but I've had little reason to complain about it).

Seems very complicated. Why not just host the CSS files statically?

My website, niflet.com, is written in Go. It uses Go's web server. While in learning mode for Go I saw that others were using Nginx instead. Now that I've gone through the development process I can see why. Go's web server is basic. Things like compressing files, setting header values and other niceties you have to write yourself or look for code written by contributors [1]. It's not as bad as it sounds though. Go has a function for serving static files like CSS files, which I wrapped in a function to serve a zipped version if the client supports that.

If I had it to do over again I would still use Go's web server, as it seems to work great. My site can conservatively handle 1 million page views hourly, using a cheap box.

[1] https://groups.google.com/forum/?fromgroups=#!topic/golang-n...

I created a project for compression doing automatically doing compression and supporting a simple proxy system [1]. It is a pretty simple system but allow for dynamically doing proxy stuff and the compression is amazingly fast. I have found Go amazingly fast and stable. It being fun is just a bonus to code in.

[1] https://bitbucket.org/lateefj/httphacks

It's funny that they argue that they dumped the JVM derivatives like Scala, but wouldn't exactly tell why, nor support their argument with any sort of data/logs.

I am genuinely curious to know why they chose Go over Scala. If it was the syntax, etc. I can partially agree because it's one of scala's weak points, but then they pitch the main reason citing performance, so I'm genuinely curious to know.

He mentions JVM memory usage.

I think he edited it after me and many other posted similar comments, coz I didn't notice it the first time...

I got the same feeling from the odd dismissal of Erlang.

Exactly, you know, my advice would be to take these blog posts with a pinch of salt. There is something called as the 'mob' mindset. The mob in general can be easily manipulated to believe in something that is not true. (Remember Julius Caesar?). For example, a year or two ago we saw the whole world of start-ups adopt MongoDB with so much vigor and almost everyone started writing "Why we moved away from MySQL to MongoDB". And two years later, we have now a bunch of posts saying the opposite - "Why we moved away from MongoDB..", etc. The same thing applies to Node.JS and Go too. They are still new technologies, so I think they deserve some time to be tried and battle-tested, instead of writing zero-data driven blogposts like these at an earlier stage. I would have been much happier to have read something like "After x years of using Go, Erlang and Scala, here is our comparison on what works well and what doesn't and how each one of them perform under different conditions" instead of "Hey we just reduced our server count to 2 using Go, hence it's better than everything out there.."

Personally, just like you, I think these guys could have achieved more with Erlang or Scala, but given the fact that they chose Go because it works well for their architecture, I am not complaining.

"We decided to rewrite the API" and now it is over 300x faster. Go had nothing to do with this, it was obviously some boneheaded original design. Probably something like fork+exec on a Ruby VM on each request to enter a chroot.

I know HN has a hard-on for Go but come on.

Are you the same asshole making throwaways to shit on all of the Go postings or are there actually more than one of you going around making assumptions, accusations and spreading falsehoods about Go.

I can't get over how pathetic it is to make a throwaway for that comment, I'm more embarrassed five people validated your comment and attitude.

Alternatively...

why we needed 30 servers: Rails

I've read this domino-effect on server clusters on several HN postmortems and I've seen various flavors of this on our own web servers. I'd like to think there's a simple configuration in most servers that prevent 100% cpu utilization from taking place and preventing the server from telling its cluster that it's still alive. Anyone have any experience with this?

The problem is queueing at the worker end and not the proxy side.

Using a fair queueing system (such as Nginx HttpUpstreamFairModule) can limit the number of concurrent requests sent to workers so they don't get overloaded. It will make requests queue up at the proxy and then get distributed evenly.

See: recent discussion on problems caused by Heroku's random order load balancing.

in linux cgroups (control groups) allow you to allocate resources to groups of processes, which makes it easy to ensure things like ssh or npre or your load balancer ping (as long as you run it out of band) still have cpu/mem even when your main server cgroup gets pegged

It's not really possible without adding resources (ie a new server).

The problem is, once your servers get saturated, requests queue up, processing slows down and users start to hit refresh which artificially and exponentially increases traffic. Then to compound things, servers buckle, which means you lose a resource right when you needed it the most.

A domino effect is one analogy, another might be how a small hole in a damn can buckle and blow open from the force of the water pushing through.

> It's not really possible without adding resources (ie a new server).

That's not quite true. With the right architecture, performance will degrade gracefully. You'll get slower and slower responses as the load goes up, and eventually start dropping requests, but the servers will not die and there will be no cascading failure.

One way to achieve this is to make sure the queuing happens at the load balancer, and no large queues are allowed to build up in the individual application servers.

I think it really has more to do with a lack of spare capacity - each server is so maxed out that it can't handle a 1/n increase in load when one server goes away.

Can you describe your workload a bit as well as any benchmarking you did to determine (and perhaps optimize) hotspots in the ruby code before you started the port?

Personally, I've been interested in moving to Go on a python-based project of mine. Thus far, I've avoided it because 1) the extra work required to self-implement a few third party libs I rely on and 2) I've been able to eek out sufficient performance using c extensions and cython.

This first rewrite in Go was for the IronWorker API so all the operations are here: http://dev.iron.io/worker/reference/api/

There were less endpoints back then, but you can get the idea. The most heavily used operation being queuing up tasks/jobs.

Science and engineering from Bell Labs behind Go is what make this possible. It is not a "different language", it is carefully selected design decisions and ideas behind it.

It relies on a principle of being good-enough, not to stuff everything in, as a "feature sellers" and "buzzword shouters" used to do.

There are also lot of work of great minds behind Lisps or Erlang, and same principles in a foundation.

"So then the decision came down to which language to use."

I find this tendency to constantly jackknife between talking about languages and talking about frameworks a little dizzying.

Shouldn't the decision have been what web stack to use? I'm sure you could get half way there with a bespoke Ruby web stack, maybe built on top of JRuby or with a sprinkling of C extensions. You could get the same poor scalability with some badly designed framework on top of Go [1].

There's a lack of meat in the post (hence why people are calling it out as PR). How do these features of Go make it easier to write applications or frameworks that cater to this particular scalability need? Could extra effort be put in up front to get something similar out of Ruby or any of the other languages, or will they always be sub-par? What is it about Go that makes it light up the rest of the stack in a way other languages don't?

Or have you just traded one trendy technology for another because it promises to be the magic bullet for your current itch?

[1] This isn't saying Rails is badly designed.

Hey that's my photo of the GOpher! :)

[1] http://www.flickr.com/photos/jianshen/8080852738/in/photostr...

Monitor Stands!!

[1] https://plus.google.com/photos/100264837158995116368/albums/...

Hah, nice photo! Are you ok with that? Found it on google images.

It's cool. :)

:) I don't know if the small plastic or larger plushy ones are cuter. Love having them on my desk.

We got a bunch of the little plastic ones sitting behind my desk: https://plus.google.com/u/0/101022900381697718949/posts/1Dbe...

+1 for the cool monitor stands but geez... Invest some $$$ in a real keyboard: scissor switches are the most painful type of keyboard switches ever : (

The Apple wireless keyboard is the best keyboard on the market right now IMO.

OK, I am sold on Go as my next language for web applications. Unfortunately, I can't throw away the existing PHP/MySQL code. Any suggestions on how to integrate Go and PHP into a single web server? Should I be using FastCGI for Apache to call the Go programs?

Out of curiousity, I would be interested to know which version/patches of ruby you were using. With ruby I'm used to running out of memory on servers long before running out of CPU. Of course, most of our workloads are IO bound, so that may be the diffent.

This was on Ruby 1.9, 2010 era. No patches. Memory was definitely a concern and probably contributed to some of it. Just to note though, after a machine was taken offline by the load balancer, it would generally come back to life.

Memory "probably contributed to it"? You don't even know why the old version was slow but you want to credit golang with making it fast? Hype much? The lack of critical thinking here on Hacker News is astounding.

When running I/O bound workloads you really should be using a multithreading-capable app server. For example, if you're using Unicorn then that is extremely bad for I/O bound workloads (the Unicorn website's Philosophy page documents this under section "Just Worse in Some Cases": http://unicorn.bogomips.org/PHILOSOPHY.html). On the other hand, something like Phusion Passenger Enterprise 4 with multithreading turned on is excellent at handling I/O bound workloads and reduce your memory by a significant factor.

    while true; end

What changed in the architecture? It is rather interesting that by just having a language change you could remove 28 servers from the system.

My guess is that removing 2-5 layers of abstraction can get you pretty far along.

We're writing most of the systems for our company in Go, and of the many reasons this ranked highly. Just removing the multiple unnecessary abstractions gets you a hell of a lot of simplicity and performance back.

Abstractions are not necessarily slow. The BSD socket API is an abstraction over sending raw TCP/IP packets to your Ethernet card, but you're not going to get a whole lot more performance out of accessing your Ethernet card directly.

I chuckled at "just a language change".

What could be more expensive than rewriting all you code? For many Rails web apps the cost of additional servers is far below the cost to rewrite things in Go.

The cost of running 28 servers is not negligible compared to developer time. If a server goes for, say, $150 per month that's $4200 wasted each month. And contrary to the rewrite, the server cost is perpetual, so the rewrite amortises eventually.

Obviously, the economics of a rewrite depend very much on the size and complexity of the code base, but it didn't sound like they had a huge code base at the time of the rewrite.

Thus the improvements where not a result of using Go (which is is a fine language). The improvements came from replacing a bloated system with a leaner one. You can do that with many other languages. There is no magic in that.

It would be interesting to hear some of the reasons you went with Go vs Clojure (and also the other languages you mentioned)

I know I'm late to the party, and no one will read this, but ...

    "... Java derivatives like Scala and Clojure ..."

Someone help me out here: In what way is Clojure a Javav derivative?

Perhaps in the sense that they are languages that target the JVM and that rely on interoperability with the Java standard library to bootstrap people making apps with them?

I can't see how it could mean anything else, but I find that, well, bizarre. Oh well, different point of view, I guess.

> with Ruby.

Title should have been: "How we went from 30 servers to 2: Replacing Ruby"

How about BDD, test-driven development and quality insurance? Does Go provide an ecosystem that supports agile refactorings that are common in lean startups?

You can say anything bad about the performance of Ruby and Rails etc. but rspec, cucumber, capybara, vcr, factorygirl are really important features to start from zero and reach a viable product.

> test-driven development

`go test`.

> Does Go provide an ecosystem that supports agile refactorings that are common in lean startups?

Go has a compiler and is staticly and strongly typed. Which automatically makes it at least an order of magnitude easier to refactor safely.

Not really sure about the other stuff you mentioned. I've never even heard of BDD before.

I've heard this argument before e.g. we don't need to test because the compiler catches errors. The compiler catches the mot trivial of error conditions. Unfortunately it doesn't address faulty logic/behavior.

I rather like what go has set up for test's. They made it as small and functional as they could.

Out of curiosity. How does a language specifically support "agile refactoring"? What features permit (or hinder) stopping every couple of weeks and removing the accumulated WTF's in the codebade?

Want to buy a left handed stapler?

Go's interfaces make it really, really easy to change things with very little code changing. Go's testing tools are superb.

Can you blog about it in the future? Couldn't find good resources and tools about testing in Go. Especially with concurrency this seems to be very important.

Go doesn't have any mocking framework like Mockito/Java or Rspec mock. You will end up writing a ton of code yourself if you want to mock something simple.

https://code.google.com/p/gomock/ works well.

Uh what? No, that couldn't possibly be less true. Have you ever heard of "interfaces" in Go? They're kind of an important feature.

Uh I think I love it. Easy to learn language and awesome performance.

I wonder how far they could have got going more of a bare-bones Sinatra app using JRuby. It's still not nearly as fast as Go obviously, but take away Rails and some of the more dynamic usages of Ruby on a faster VM and I'd venture to guess you might have been able to go from 30 servers to 10-15?

That's still not 2 mind you...

There's simply no technology that makes you switch from 30 machines to 1 (you say you keep 2 just for reliance).

Obviously there must be other architectural changes in play here (I'm guessing RoRs thread-per-request vs some kind of event loop on Go?). Please be more specific.

Go is good at keeping all the cores on the machine pumping. Maybe they switched to machines with more cores.

Why Go over VisualBasic?

There are a lot of things that can be said about the "average" programmer in such-and-such languages, what they do and don't do, what habits they have, etc. But I'm not interested in working with merely average people.

That's right. People like you and me work with the top 5% of programmers! It's because I'm smrt.

You have an error in Opera

'SyntaxHighlighter.config.bloggerMode = true;'

Undefined variable: SyntaxHighlighterError

Go 1.1RC just around the corner in April. A lot of good stuff in Go 1.1RC.

If you look for a Golang programmer today, it's more likely you'll find a true programmer instead of a me too programmer. So if you're a true programmer, learn Golang. ;)

Now I know that the dynamic languages such as ruby and python are not known for performance, but the benchmarks I have seen with Go look like it isnt exactly the fastest language available either. After reading this article I am more apt to think that something is/was VERY wrong with their Ruby code than to think that Go is that much better. Maybe my lack of experience with either GO or Ruby is showing here, but Ruby cant be that slow can it????

p.s. come disrupt our business: with nearly no hardware, no costs, and no maintence burden you too can pass OMG huge amounts of message traffic.

this is one bold as fuck post.

http://www.dartlang.org/support/faq.html

>> Q. How does Dart relate to Go?# Dart and Go are both language projects started at Google, but they are independent and have different goals. As a result, they make different choices, and the languages have very different natures, even while we all try to learn from each others' work.

Hmm, don't think I want to bet the farm on that.

Don't want to bet the farm on what, do you mean Dart or Go sticking around? Go is used extensively through Google. It powers a significant DB layer that powers YouTube. It serves up all of Google's static downloads and several employees hint at other significant projects they can't speak about.

I've only dabbled in Dart, I can't speak to how vital it is, currently it only seems viable when translated to JS or when using in a server-side VM, which is certainly an option if you want.

(sorry, I originally had written Google where I meant YouTube, see "Vitess" for more info)

We decided to rewrite the API. This was an easy decision, clearly our Ruby on Rails API wasn't going to scale well and coming from many years of Java development and having written a bunch of things that handled tons of load with way less resources than what this Ruby on Rails setup could handle, I knew we could do a lot better

Was anyone else expecting this to be a boast from EA about how they cut their operating costs for the SimCity launch?!?

man, you just made me worry very much now - we are making taxi dispatching api using ROR - and we will lunch it soon

Why worry? If you can handle the initial load, you can still optimize later on, rewriting some functions or parts of the system with something faster.

great

don't worry... our Ruby servers kept us churning for a long time.

If perf is that much of a worry, you can look at JRuby, that will get you a bit (or a lot) farther from what I've seen.

I wish Go has named arguments like Python and Scala. Reading and debugging code could be so much simpler.

Why Go over Erlang?

(Note: I'm a big Erlang supporter and 90% of my startup's code is written in Erlang)

Erlang is not memory efficient (compared to Go? I don't have benchmarks to back this up) AND the learning curve for developers not exposed to functional programming (which is a lot) is quite a bit higher than it is in Go.

Hmm, wouldn't it be better to rewrite the ruby services to Go, the actual workers and not the entire API? I don't think the API HTTP requests where your main issues here...

Would be great to hear more specifics of the rewrite. What did the API do, what pieces were built in RR, what got replaced in Go, etc.

At least as much as you're willing to share. :-)

The problem was Ruby, not Rails. Did you give ever give Jruby a chance?

Rails is not for API,why not use eventmachine?

Why was Go chosen over NodeJS ?

Mostly because I can't stand Javascript. It makes me cringe just thinking about it. Go is much nicer to work with.

Hey OP! I appreciate your sharing. Since you came from ruby and we're on the topic of the language itself, I'd appreciate your impression of how well Go supports collections. Is there or could one write something like http://underscorejs.org/? Can you do this kind of thing?

[1, 2, 3, 4, 5].reject {|i| i < 3}.map {|i| i + 9}

I did the go tutorial the other day and I became a little worried that one would not be able to do this kind of thing without a bunch of unwieldy declarations. I think that would be a showstopper for me.

Thanks for any insight you might have!

Although I personally like CoffeeScript, I must say that Go fills me with joy for some reason.

It feels "fresh" and lightweight to the beginner, and reminds me of TurboPascal in some ways.

Why would NodeJS be the most obvious alternative to Go? The two seem more dissimilar than similar.

No Love for Lua and LuaJit.

Man, what is it about Go that just brings out the pricks of HN?

There are over three throwaway accounts created exclusively for this thread to shit-post about Go, to steal an old term I have never, ever used on HN before.

I'm not exactly sure, I've noticed the same thing.

Go appeared a few years ago, quickly became pretty well-known, and continues to grow in popularity. Maybe it's seen as "threatening" to older projects like Ruby and Python? The most vitriolic comments always seem to come from Ruby, Python, and Rust partisans. Of course, I'm not sure why they're so threatened by it, they themselves explain in great depth how the lack of generics or exceptions will prevent anyone from writing any real code in it, or even trying to in the first place!

great stuff!!

Reply on news.ycombinator.com