Comment by grey-area | Hacker News Reader

Comment by grey-area | Hacker News Reader

Parent Full thread

Great post. I've done some experimentation with Go for web dev, and encountered similar problems; it's such a delight to write code in that I'd love to use it in production, but can't justify spending a lot of time debugging the immature libraries and writing new ones, particularly not on the clock for clients. I too ran into a problem with null types in another library interfacing with postgresql. The weak points I found were:

* Lack of an ORM (there are many, almost all incomplete, and lacking in some way)

* Immature or incomplete db interface libraries

* No db migrations (would love to see a simple sql-only solution here)

* Package management is simple and elegant but without explicit versioning, so forking is the end result to ensure stability

* No process pool management for running Go behind something like Apache or nginx.

This goagain tool looks interesting for the last point though:

https://github.com/rcrowley/goagain

and the routing I found pretty straightforward with something like github.com/gorilla/mux so there are solutions, they're just not always as fully baked or as mature as you'd find in other ecosystems. One place Go shines though is the great built in web server which really simplifies getting up and running and testing, and which I see they've built on here.

Given how easy and pleasurable it is to work with, and the focus on practicality rather than language features, I'm quite confident that Go will reach it's stated aim of becoming a popular server-side language quite soon. For those new to the philosophy behind the language, I found this informative - http://talks.golang.org/2012/splash.article

I went on a framework binge over the past two weeks to see what the state of web development was. I looked at a dozen or so frameworks including Go frameworks. I came to the same conclusion you did.

Revel was the most promising Go web framework that I saw though very immature.

Go ORMs aren't ORMs. They are object persistence frameworks (at best). None of them really provide the "R" in ORM. Most of them basically do "SELECT * FROM table WHERE id=12" and dump that into a struct. There goes the "M" as well.

The Go language itself is very nice to work with though. It's a great language. Give it 2-3 more years and a truly usable ORM and web framework will crop up.

Many people coming from dynamic languages rely too heavily on ORMs. It's not very hard to write some SQL that will do what you want it to do, and it'll almost always be faster than an ORM. There is a little more setup, but it's not really that hard unless you have a really huge data model. Also, many people are moving away from relational databases for web platforms anyway, and non-relational databases are a lot easier to write code against. Check out the mgo package for running against mongodb.

I don't think people use ORMS because relations are hard, but because they are boring, repetitive, and just complex enough to make them tricky but not interesting. So ORMS are used to take away some of that cognitive load when you just want to say that Story has_many :commenters, via: comments or something similar. It's simpler to express what you want, gets rid of boilerplate joins etc, and usually you can drop down to SQL if you need it to fine tune a query, so why not?

Re relational versus non-relational, they're really suitable for different kinds of data, and it's disingenuous to suggest that one is the future and one is the past.

This; the moment I realized I was never going to choose to do CRUD web stuff in Go was when I realized that this was the perspective Golang developers have on ORMs.

That's funny... knowing that ORMs aren't everywhere actually makes me more likely to use Go. I still have Hibernate flashbacks.

Hibernate is not a good example of an ORM, frankly. (I myself have horrors of both Hibernate and TopLink, which was the top Java ORM way back when.)

Ruby's ActiveRecord is a much better choice. It has an excellent balance between SQL and OO. It doesn't pretend that SQL doesn't exist; on the contrary, it encourgaes SQL use, and merely maps tables to objects, adds a bunch of useful features (data validation, change management, automatic joins, declarative migrations) and gets out of your way most of the time.

For example:

    users = User.where("created_at > ?", Time.now - 1.year).
      order('name').limit(10)

Results are lazily loaded and composable, so you can do:

    users = User.where("created_at > ?", Time.now - 1.year)
    if (page = params[:page])
      users = users.offset((page - 1) * 10).limit(10)
    end

Joining is easy:

    users = Users.joins(:accounts).
      where(accounts: {type: 'facebook')).first

becomes something like:

    select users.*
    from users
    join accounts on accounts.id = users.account_id
    where accounts.type = 'facebook'
    limit 1

Have you used rails ORM (activerecord) or sqlalchemy ? (I haven't used the latter much but people rave about it). Hibernate is an ORM taken to the extremes

At first when I started doing Android apps I was writing raw sql code. I very quickly remembered how tedious and repetitive this is and went running for the nearest ORM. Nothing feels better than ripping out huge chunks of boilerplate.

Personally I don't like ORMs because they're black boxes. When some complex model relationship results in crappy generated SQL (which I've experienced a number of times) I had to sit down and write SQL. The problem being that after a couple of years of ORMing my SQL skills had atrophied.

So now I've gone fully circle: from SQL to ORM and back to SQL.

for migrations in go, check out goose (https://bitbucket.org/liamstask/goose). I've used it, and it's pretty simple, but works pretty well.

You can write migrations either in plain SQL (my preferred method) or in go.

Great. Thanks for the link. That's one thing I can cross off my list hopefully.

No process pool management for running Go behind something like Apache or nginx.

Aside from goroutine scheduling issues, what would be the main reasons to run multiple Go processes, instead of just running a single multi-threaded process?

There are perhaps a few advantages to having lots of processes that you wouldn't get in a single threaded app: redundancy in case a process does hang, rolling restarts to switch out to a newer version of the code seamlessly by starting multiple processes to handle requests before killing the old one, and I suspect you'd hit some limits of the scheduler, as you hinted in your question.

It's probably possible to have most of that logic in a single Go process which runs a bunch of sub-processes for serving requests, but then you're also pretty much writing a load-balancer as well as your app each time. Perhaps better to separate out those tasks and put them into a separate process manager which runs a pool of processes, as on other platforms like Ruby with Unicorn or Passenger? Those platforms have other reasons of course for scaling with processes and not threads, which don't apply to Go.

Not sure how hard or efficient this would be (just using one process) as I haven't tried an app in this style in go, have only been playing with it so far. I would be really interested to see a Go server implementation that managed a bunch of goroutines to serve requests, are there any examples out there?

> Not sure how hard or efficient this would be (just using one process) as I haven't tried an app in this style in go, have only been playing with it so far. I would be really interested to see a Go server implementation that managed a bunch of goroutines to serve requests, are there any examples out there?

I'm slightly confused by this question, because that's what the standard library does. If you've ever used net/rpc, or net/http, then it spawns goroutines for each request (or connection, respectively).

If you meant to say spawn a bunch of _processes_ to serve requests, then no, I don't think anyone has done it. I don't think it makes a whole lot of sense for anyone to write code to do this in Go, tbh.

> redundancy in case a process does hang

If you're talking about deadlocks, then only the deadlocked goroutines will be blocked. The rest will make progress, just as if you had multiple processes.

> rolling restarts to switch out to a newer version of the code seamlessly by starting multiple processes to handle requests before killing the old one

You can do this with only 2 processes, old and new. You can spawn the new one, tell your LB to add the new to the pool, wait 30 seconds, remove old from pool, wait 60 seconds, kill old one. You don't need an LB, you could, of course, use an nginx frontend or something instead. There's also some neat ways to do nginx-style zero downtime restarts that I've never tried, but heard good things about.

> I suspect you'd hit some limits of the scheduler

AFAIK, limits of the scheduler tend to be hit when you increase GOMAXPROCS to something above 8. At this point, you'll spend a lot of your time in the runtime managing goroutines. My solution is just to run multiple processes with GOMAXPROCS=8 and point your LB at both of them. Again, you can just use nginx.

Feel free to experiment with the model you proposed, but this is relatively non-idiomatic, and the context-switching cost will start to mislead you as to Go's actual potential. The advantage, btw, that you get when you use the model I spoke of is that in memory caching, connection pooling, context switch time are all close to optimal, and you have fewer processes to monitor/restart/update.

Thanks for the reply, which clears up some of my misunderstandings (sorry, new to Go), I was using the standard library and hadn't looked under the hood, I'll go take a look at what it does. Re rolling restarts:

You can do this with only 2 processes, old and new. You can spawn the new one, tell your LB to add the new to the pool, wait 30 seconds, remove old from pool, wait 60 seconds, kill old one.

You still need a LB to do this, though I take your point that you could use nginx, might experiment with that.

Reply on news.ycombinator.com