Why I chose to write Nuuton in Go
nuuton.com
nuuton.com
I chose Go because it is being developed to solve the same problems I have to solve. All by a group of scientists an programmers that are each twice as smart as I have ever been (on my best day).
So, what are the problems you are trying to solve? And why can Go solve them better than, Java, Erlang, C++, Scala, or any other language?
If you could answer that question, you'd have an interesting blog post. And if the answer is 'concurrency via Goroutines and channels' (it always is), explain why they are better than Erlang or Akka's actors, STM in Clojure or Haskell, or other proposals, and you'd have a stellar blog post.
But really, this at the top of Hacker News?
They aren't. I analyzed Go for my personal use, and I have a high degree of familiarity with Erlang and Haskell. The conclusion I came to was that it is a step back from Erlang and Haskell's concurrency support, and that I didn't care to step back into those problems. It's easier to do concurrency correctly in Go than it is in Python or C, but it's still pretty easy to make it blow up in your face and take down your whole program. You to both learn Go, and then learn all the implicit (and to some extent still forming) rules about exactly how you can safely handle channels, which the compiler can't help with.
(I tried to keep an open mind, but I'm still of the opinion Erlang's processes with asynchronous mailboxes is better than Go's synchronous channels. Note that's two differences, not one; sync vs. async, and mailboxes vs. channels. And they are effectively synchronous, even when you set a buffer size, because there's still a hard cap on when the channel is full; semantically the difference between a 1-element sync channel and a 10-element sync channel is pretty minimal, you get the same crashes and failures either way if you misuse them, it just takes a bit longer in the 10-element case. Oh, and since "synchronous" seems to have had its meaning mangled here in the past few years, in this case it means that the writing and reading of a channel must coordinate.)
But if you're coming from a traditional Algol language, it's a big step forward. I'm sure coming to Go from Python or Lisp feels like a blast of fresh air, because it really is an improvement. And it gives you this without having to abandon your Algol roots, which most of the rest of your alternatives do (except Akka, though I don't how hard those are to use correctly since they're in a mutable language), and that is saying something.
You hit the nail on the head. It does feel like a blast of fresh air. I've been eyeing Erlang and Haskell, but have not had to chance to do something worthwhile with them.
My personal experience base is freaking weird, and if you don't have Erlang or Haskell already under your belt, then I recommend considering Go quite strongly. You won't be bothered by some of the things I'm bothered by.
And I will find some fresher story to say this on at some point, to be fair.
Which IDE would you recommend for it?
If you want to argue that Emacs is not the IDE because it doesn't has feature X, then please not that it has great integrated internal environment. For Go "IDE" features check this: http://dominik.honnef.co/posts/2013/03/writing_go_in_emacs/
Vim has similar features.
Go vs Haskell/Erlang has a similar showing in TechEmpower's Web Framework Benchmarks: http://www.techempower.com/benchmarks/
Indeed, that's one of my frustrations with Go, one of the big reasons I chose to leave it alone. The ways in which it is much less safe to use concurrently than Erlang or Haskell aren't performance related, they're related to the choice of primitives. (In particular, I can't see how to build an OTP equivalent for Go. This is a big problem for me, since I was looking at it to replace Erlang, but I'm not ripping out something that has demonstrated how bullet-proof it is to replace it with something more fragile like Go, when performance is not my problem.)
To be clear, I'm just talking about the concurrency primitives, and I do not believe that Go's concurrency primitives are better than Haskell or Erlang's, full stop. That does not mean Go is not a better language for some tasks, and I further believe Go's concurrency primitives are a step up over most current "conventional" languages, equally full stop. For one thing, to program in Erlang or Haskell requires programming in Erlang or Haskell, which is why I mentioned the fact that Go is still an Algol-esque language. That's a legitimate advantage. I've paid the price of learning the alternate paradigms, but I would generally recommend Go to a larger corporate deployment before I'd recommend trying to convince a couple dozen people to learn Erlang. So much... whining.
Sure but that is why I brought it up. Performance and safety are very often two sides to the same tradeoff. You were saying you thought the concurrency of Erlang was safer, and I was proposing that depending on the problem domain, a little risk can be worth the performance gain. That being said, I think Go makes concurrency easy to get right (much less risky than say C/C++), it just doesn't ensure correctness.
>>...Go is still an Algol-esque language. That's a legitimate advantage. I've paid the price of learning the alternate paradigms, but I would generally recommend Go to a larger corporate deployment before I'd recommend trying to convince a couple dozen people to learn Erlang.
Wise man :)
Haskell is pretty much the counter example. But Haskell will always to be weird to most imperative programmers. This is why I am far more exited by Rust than by Go, it promises to deliver performance and safety in an imperative language.
Also, there are actor implementations on top of languages that are more performant than Erlang. E.g. see Akka for Java and Scala.
I said "very often" for this reason, there is always an exception to the rule. I do love Haskell too.
>>This is why I am far more exited by Rust than by Go, it promises to deliver performance and safety in an imperative language.
To be clear we were talking about concurrency safety, not type and memory safety; Go is type and memory safe. If an organization using Go chose to tell programmers they could not send pointers between goroutines or share global state, Go would be just as communication safe as Rust. Just in Go, this communication safety is optional.
Not if you're using parallelism (GOMAXPROCS > 1).
> If an organization using Go chose to tell programmers they could not send pointers between goroutines or share global state, Go would be just as communication safe as Rust. Just in Go, this communication safety is optional.
It is in Rust as well. You just have to declare that you're opting into data races; safety is the default and enforced by the compiler if you don't opt in.
Unless I'm mistaken (and please correct me if I am!), synchronous primitives can be used to implement asynchronous primitives, but the converse isn't true. If that's the case, why would Erlang's asynchronous primitives not be inferior to Go's synchronous primitives?
Many servers solve the c10k problem (not that it is an issue for many users anyway), but it always seemed as much interesting to me how your code is structured with a highly concurrent server. E.g. Yesod and Snap (via WAI) allow you to stream the request and response with conduit (and enumerator) and build nice and composable pipelines. Play 2 is also moving in that direction with their iteratee implementation:
http://www.playframework.com/documentation/2.1.1/Iteratees
Iteratee/pipe-based processing would never be possible in Go, since it does not offer parametric polymorphism and typeclasses. Sure, you could write transformations to take and produce interface{}s, but it will give you as much type safety as a dynamic languages and type switches in every function.
As in similar to the above benchmark, as in spanked by Go in Erlang's case.
I am looking forward to see your search innovations...
Do you remember when Rails and Node were every day on HN? Now it is time for Go.
There should be a lot more text in that post to answer these important questions.
Compared to X, Go has Y:
Java: Less verbose, faster builds, native code (no JVM needed), cleaner OO, structural typing, better memory usage, structural typing, more convenient concurrency
Erlang: Better performance, better memory use, cleaner syntax, easier to train people on/hire for, statically typed
C++: Type and memory safety, much simpler syntax, better build times, large standard library, structural typing, garbage collection, easy and safe concurrency
etc, etc
Of course using an offbeat stack is going to hurt - doesn't mean you have to give up Lisp too.
A decision made odder still since being an independent project, one would think maximizing individual productivity would be the only hope of yielding anything novel and useful in a field like search that isn't a fitting of an off-the-shelf to a business problem. (Something I've done before, it was quite neat.)
Go is not a language focused around individual productivity in any capacity or scope. It's the language of pyramid-building Egyptian Pharaohs. (Throw 1,000 lives at it, it'll get 'er done...)
I say this as someone rather envious of any language that has a native compiler that builds static binaries. (Oh, OCaml...)
As we ponder on what orangetide is up to, lets keep in mind he's not explained or revealed anything at all about Nuuton and for all we know - could be having us on for the sake of self-promotion/promoting Go.
The incorrect speculation about the origins and purpose of Go make this even stranger.
Go exists to make people like Rob Pike happy. The Plan9 and Suckless community of programmers. Those types. That it happened to solve the problems of some Google programmers who were making it was almost incidental.
Well, you are right. I am promoting Nuuton. After a almost a year of working on it and not speaking much about it, I decided to start opening up. I don't want for it to fail due to being too secretive. Time will tell if this is the right decision. But I'm having a great time building in Go. If all fails, I will still have the knowledge of doing it.
Edit
I also considered Clojure. Even spoke about it to some people on this board. But it seemed to take me along the same path Lisp did. Still a fine language, though. I have nothing against it. It did not make much sense in my mind.
My own path was GW-Basic -> C -> Common Lisp -> C# -> Python -> Clojure, with other languages learned along the way.
Seems to have been my impression as well. Lisp with its macros can make an individual programmer who doesn't need to worry about coding conventions incredibly productive. I see Rust has a semi-powerful macro system too although not to the extent of Lisp, as it doesn't look like you can evaluate arbitrary code at compile time.
Even did some coding in Go for a few minor projects.
But that experience showed me that it actually felt like a time travel back to 1992, when I was using Native Oberon.
Sure the language feels like a better C and I find it good that it has native code compilers as the default implementation.
But since 1992 we have gained lots of abstractions in mainstream languages that it is pity to throw them out of the window.
I don't know the motivation behind the people developing Go, but I found it very easy and convenient to work with it, almost as easy as with Python sometimes, which is mind-blowing IMO given the static, compiled, strict, etc. nature of Go. So again, don't know whether the guys behind the language were motivated by personal productivity, but they have definitely achieved it.
Though I didn't really try Lisp other than quick tutorials, so I can't make a comparison with that.
Not really.
For one, Google didn't develop Go (it wasn't a company decision to get a new language, as Java was for SUN, .NET was for MS and Dart was for Google). Go was merely started by some a team working at Google as a side project and it grew a little from there. They do pay several people to work on it nowadays, but it's not some "bet the company" language. Also, Google hasn't mandated replacing any C++/Java stuff with Go, and nobody is touching their search engine code. If it happens, it happens organically and for lesser stuff.
Second, Go was developed for "systems" software (albeit in the sense of network servers and such), not search specifically.
History shows that a lot of extremely successful things started out that way.
> and it grew a little from there
That's a bit of an understatement. Go has hundreds of contributors, and is already used by thousands of companies.
> They do pay several people to work on it nowadays, but it's not some "bet the company" language.
Definitely not. For that matter, neither is Dart.
> Also, Google hasn't mandated replacing any C++/Java stuff with Go, and nobody is touching their search engine code. If it happens, it happens organically and for lesser stuff.
You make it sound like Go isn't being used at Google, which is disingenuous. No, it probably hasn't replaced the crawler and pagerank logic--it'd be silly to rewrite all of that--but it is continuously being picked for new projects internally. Some of the few such projects that have become public are Vitess, the database frontend that powers YouTube, and the code that powers dl.google.com.
(That there is no known mandate to replace existing code isn't indicative of much. It's a bad move to rewrite that much code for any reason.)
> Second, Go was developed for "systems" software (albeit in the sense of network servers and such), not search specifically.
It started out as a "systems language" (which roughly translates to "for writing backend software", but was commonly misunderstood), however that description has been removed since Go has proved useful in other areas. It is now simply a general-purpose programming language.
Also a lot of failures. But both cases are beside the point. I didn't correct him on whether Go is good or not. Just in that it was not started officially by Google.
>That's a bit of an understatement. Go has hundreds of contributors, and is already used by thousands of companies.
And yet it's still a totally marginal language in the grand scheme of things. Not even 1/100 of what something like Java or .NET is, for example.
>Definitely not. For that matter, neither is Dart.
Agreed on Dart (and I don't think I said it was). But .NET and Java were.
>You make it sound like Go isn't being used at Google, which is disingenuous. No, it probably hasn't replaced the crawler and pagerank logic--it'd be silly to rewrite all of that--but it is continuously being picked for new projects internally. Some of the few such projects that have become public are Vitess, the database frontend that powers YouTube, and the code that powers dl.google.com.
Yes, I've read about both, I follow Go since the beginning. Those two quite humble projects are the only things that have become publicly known for 2 years now. A db load balancer (which had some Go issues they had to code around even), and an abandoned part of Google infrastructure in C++ that a Go team member volunteered to rewrite in Go.
>however that description has been removed since Go has proved useful in other areas. It is now simply a general-purpose programming language.
I believe the reason it that description has been removed was because people complained that "systems software" means specifically OSs, drivers and such, not just backend software.
Not really. While Go was presented as suitable for writing classically backend software like databases, it was also presented as suitable for writing compilers, programming tools, debuggers, IDEs, and web browsers, none of which are "backend software."
The Go 1 compatibility contract guarantees that no breaking API changes will happen for many years, barring some serious security issue: http://golang.org/doc/go1compat.html If it compiles now, it should continue to compile and work as expected in the future.
If I could have a penny...
If you are seriously hoping for this to be a full time business then I would strongly recommend you read "The Lean Startup" or similar. This blind technical pursuit of perfection is a pretty classic approach for us techies and something i'm guilty of myself. It was actually that process that made me realise what my strengths actually were and what I would need to do to start a world class company (work with great people).
If this isn't really a commercial project then just ignore the above and happy hacking, perhaps you should consider open sourcing it?
I do follow the lean principle. But one cannot blindly adhere to them. Nuuton is not the sort of project where you just try and get something to work. There had to be a foundation first. But from now on, development will follow the build-measure-learn cycle. I am open sourcing parts of it. Mostly Go libraries that might result from developing it.
"[Lisp] had been my sweetheart for a log time."
Things got complicated quickly. I was writing too much code to achieve simple things. Plus I was writing a lot of one-off code. I prefer to write as much reusable code a I can. Mostly because I'm lazy and I like to keep things simple.
I'd be interested to hear more details on this. Writing non-reusable code and lisp seems antithetical to me. Do you mean the lack of libraries and interfaces to other services?