Maybe its just me, but I like the fact that adding exponential backoff to an HTTP request goes from http://play.golang.org/p/8WQ8XXrKw0 to http://play.golang.org/p/3tCiNhAfo3, in no small part due to the explicit error-handling style of Go.
228 karma · joined February 24, 2012
err := datastore.Get(c, key, record)
if err == ErrNoSuchEntity {
// entity not found
return
} else if err != nil {
// some other error
return
}
I understand your point in general, but have to disagree with the example you chose. A 404 implies Page Not Found.
The Python framework I use (http://bottlepy.org/docs/0.9/_modules/bottle.html) makes this a 500 Internal Server Error, with the text "Critical error while processing request: /foo-bar". IMO, this is ugly, and not super user-friendly, so I usually end up writing try catches around "one-liners" to log a better error than a stack dump, and present a friendlier message.
If he really wants this kind of framework-provided error message, then a panic would be more appropriate, because the HTTP server in Go will do the same thing as bottle. It's definitely not going to be as succinct, but it'll be more succinct.
Compressing headers is not for handling large headers. They are for handling repeated headers. For example, with every request-response pair, you waste a few bytes saying "Accept-Ranges:bytes", you waste a few bytes saying "Server:Apache/2.4.1 (Unix) OpenSSL/1.0.0g", you waste a few bytes saying "Connection:Keep-Alive", etc. Further, when you compress headers, you save a few bytes for repeatedly using the string "Cache-Control:", "Content-Length", "Last-Modified:", etc.
If you take a look at the work done here: http://www.eecis.udel.edu/~amer/PEL/poc/pdf/SPDY-Fan.pdf, (ironically, the pdf is entitled SPDY-Fan after one of the authors), you will find that the average HTTP request they studied is (reasonably) 629.8 bytes whereas SPDY (after some warm-up) transmits 35 bytes on average. Similarly, the average http response they studied is (reasonably) 437.7 bytes whereas SPDY (after some warm-up) transmits 63.8 bytes on average.
> Another new addition was de-serialization. But I see nothing wrong with receiving the data in the order I requested it. If the transmission is interrupted, at least then I can restart where I left off. [...] Alas, no one used it. Why? Maybe because it didn't have a catchy acronym and a corporate brand behind it.
Historically, browser support for pipelining has pretty much been limited to Opera. Chrome has an implementation now, and I believe, so does Firefox. I don't think either of these are enabled, by default, although I could be mistaken.
For issues with pipelining, this document has a few hints http://tools.ietf.org/html/draft-nottingham-http-pipeline-01.... As well, pipelining cannot be used for non-idempotent requests, i.e. non-(HEAD|GET) requests.
Realistically speaking, pipelining is essentially a hack to make a synchronous protocol more efficient. If the creators of HTTP were to repeat their efforts today, knowing how the protocol they are creating is going to be used, I'd bet my left arm that they would create a multiplexed protocol.
> especially not now.
Are you also going to avoid all the sites that use an unpatched OpenSSL? CRIME seems to affect those just as it affects SPDY.
2 theories on why gcc is doing better. Maybe its unrolling the loop, and avoids a bunch of jumps. Alternatively, it could be using SSE, but this theory is a much less likely, because I would expect it to be 4 times faster not twice as fast. gc doesn't use SSE instructions yet.
You tend not to be writing all 10 methods in a particular call chain at the same time. You will be writing a few methods that call each other inside a module. You paint this as a massive timesink, and I can assure you, it definitely is not.
Lazy programmers can also have catch-all exception handlers. I don't see how exceptions help make lazy programmers perform due diligence.
That being said, there is a place for exceptions. Truly exceptional conditions such as index out of bounds, or nil pointer dereference, or some internal precondition violated, should be treated in an exceptional manner. Go does this with panics, and panics are almost never caught as part of control-flow. They tend to be caught at the root of goroutines, logged, and the goroutine killed. The HTTP library, for instance, will catch panics in any goroutine it spawns, and write a 503.
I just find it odd that people treat commonplace things as exceptional. File open failed? Could not resolve hostname? Broken TCP connection? These aren't particularly exceptional things. They are probably not a result of a bug, and so should be handled by the programmer.
I think its much more natural for a memcache API to return an error if the server is not reachable, and I can continue to execute the current function. Similarly, I think its more natural for a "users" API to return an error if a particular user doesn't exist so I can redirect to a signup page or something, rather than throw a UserNotExistException.
And yes, errors as return values may seem to add more code to simple examples. But I find it does wonders for clarity/readability. Using "regular" control-flow for error conditions and the "happy path" makes code much easier to follow; this is as opposed to trying to intuit the different ways control can jump from the happy path into the error handling.
Also, I find that having to write the "if error return error" makes me pause to think about how to handle errors properly. For example, if the function I'm writing literally cannot proceed I will return the error. If its a really weird place to be getting an error, write it to a log and return the error. If I can ignore errors (like the memcache example above) then I keep going.
http://www.belshe.com/2012/03/29/comments-on-microsofts-spdy...
Relevant docs: http://golang.org/cmd/ld/ http://plan9.bell-labs.com/magic/man2html/1/2l
Also fwiw, it has never been my experience that json decoding/encoding become the bottleneck in a web app. I/O (to a database, or the filesystem) is by far the largest bottleneck in any app I've profiled. A good benchmark for a web app language is hard to write, because it tends to depend on (unreliable) IO-bound systems.
This discussion is getting a little off track.
I'm simply saying that dismissing Go for not supporting plugins is rather illogical, as there are a lot of use cases that have no need of plugins. Servers being a good example of one of these use cases.
Sure, if you do it stupidly, and have a highlighter process per open document or something, it could add up over time. But if you share highlighter processes between documents, you have 1 process that you keep open for the lifetime of the application. A simple HTTP server in Go takes < 5 MB of RAM in its steady state, a server using domain sockets would probably be comparable. And even better, a bug in the highlighter doesn't mean your main application crashes. If it crashes, the parent merely restarts the process. Even better, its more secure, in the sense that you can run your highlighter with absolutely no OS privileges. Also, this allows for plugins written in any language that supports sockets.
All I'm saying is that there's no evidence that IPC isn't performant enough. And IPC comes with security, compatibility, and isolation advantages, to boot.
Note that I'm definitely not saying that dynamic loading wouldn't be nice. Do I miss it, when working with Go? Not for my use cases. "Plugins" for servers don't really make sense. But dismissing Go because it doesn't target every use-case at the moment is somewhat shortsighted.
Holding up Eclipse as an example citing performance is perhaps not the best idea. And Chromium seems to do just fine (performance-wise) using IPC between renderer processes and the main process.
While it is a bummer, I'd argue that the OP is the one that does not define a trend.
- It has an "exception" mechanism that is rarely used, in favour of error-valued returns. (http://blog.golang.org/2010/08/defer-panic-and-recover.html)
- Apparently, vitess didn't need generics. You'll notice a bit of casting here in their implementation of an LRU cache, but writing containers isn't exactly the main purpose of the library. I do admit I want generics, for the sole purpose of stopping people parroting that criticism without actually using the language. (http://code.google.com/p/vitess/source/browse/go/cache/lru_c...)
- Not sure what you mean here. If you mean a Go library is being loaded, then there's an issue for that, but one unlikely to be fixed in the short-term because of Go's unusual calling convention. If you mean Go loading a library at runtime, I don't know why you'd want this. Either way, static linking seems cleaner and more self contained to me.
"Short compilation times" being available in any language is total bull. Try building chromium or firefox in a reasonable amount of time, without a build cluster. Don't you think if there was a way to speed that up, the developers would make that priority 1?
Channel/Goroutines are of course available in all languages, Turing-completeness implies that they must. However, in practice, does all code in those languages make use of the same concurrency primitives? Do they read as nicely as Go does?
When referring to a friend named "Edward" in a text to another friend planning Edward's surprise birthday party, you can probably refer to him as "Ed" or "Edward". You definitely don't need to refer to him as "Edward (Parent's SSN:12345...)" or "Edward (Philip's Son)". While these latter forms are less ambiguous (and more searchable, to boot), the context is more than sufficient to disambiguate.
The fix is well-known (a precise GC), just implementing it hasn't happened yet.
In order to pull of an attack, you'd need to know what address range the program in question has been allocated, then figure out the smaller range that the runtime is actively using, then give it data with integers in that range. This is impractical.
If you think you can pull it off play.golang.org lets you upload text to a Go program on Appengine, then it compiles that program and runs it. This gives you 2 programs to attack, the playground binary, and the one compiled from your source. If you can do it, you'll have a way to kill machines inside Google.
Good luck.
If you think you can still do it, all of *.golang.org and golang.org are running Go on Appengine, with the source code being freely available. This is your opportunity to get a back door into Google's servers.
About this issue: yeah, its unfortunate, but its a property of the class of garbage collector that Go uses atm. My servers are all 64-bit, so doesn't really affect me. But I do feel bad for those who are trying to run Go on ARM, or on 32-bit servers.
Let me answer some of your points more specifically: NUMA is a concern, but with some work on the schedular to give goroutines slight affinity for threads, this can be largely mitigated. This could be as simple as a scheduler policy like `take the first queued goroutine that previously executed on this thread, looking upto 5 into the queue, otherwise take the first one` instead of `always take the first one`. The difficulty with this strategy is you could experience starvation of goroutines, and there are a ton of other complexities, which is why the current scheduler is so simple. I believe this will get better...
Context switching isn't the issue with threaded implementations of servers. Writing a server with a thread per connection is a bad idea because of the memory requirements. 1000 connections will lead to gigabytes of memory in use.
I don't think Go's performance envelope has shrunk or become less predictable, IMO. I think what we've given up is control over what threads execute what goroutines, essentially, the NUMA argument above. This will hopefully get better with time.
The nature of the problems we are trying to solve, and the hardware we are solving them on.
> But yeah in general the C interop is very simple in most languages.
You're not taking into account the fact that the C code cannot be allowed to block the main loop of the program; as is true in any event driven system. Yea, other programming languages might not have to deal with concurrency issues, either by ignoring threading, giving up on an asynchronous system, or giving up on C interop; none of which seem like a worthwhile sacrifice. What is your problem with this implementation technique? It's performant in practice, and its not in your face when actually writing code. This is not complexity you have to worry about. It happens "behind the scenes" as it were.
Go might be challenging to search for, but you're not writing search queries, you're writing posts about programming. It's pretty unambiguous as far as I'm concerned.
And fyi, I've never downvoted you. I think with the increasingly desperate tone in your writing, its not hard for people to realize that you're just hating on Go, and at least the responses to your misinformation are educational, and often interesting.
Of the links in that list, 2 worked for me, and both were from 2002. Not exactly up to date information. Also, Erlang, does something similar w.r.t. to multiplexing multiple processes onto fewer threads, so Go isn't exactly alone in this regard. Even Twisted, and Node use a weaker form of this, where everything runs on one thread.
> Read the comment at the top. Hardly trivial.
First, the comment's meant for language implementors. You don't need to read this comment to be able to do C interop in Go. Second, this isn't even that complicated... did you imagine C interop with other languages is done in a nice, simple way? Any language needs a way to translate calling conventions from the source to the destination and back when doing interop.
And, yes, please stop calling it Google Go. People hardly ever say Microsoft C# or Sun Java or Apple Objective C, or Ericcson Erlang, or Netscape Javascript...
> These guys do testing to confirm their assumptions. This is how any science gets done... hypothesis, then experimentation
As to your comments on how compression works, I admit my lack of knowledge on how lz works. But I trust the results of the experiments done by the guys at the University of Delaware. If you have a better dictionary in mind, take their sample data, run the same experiment with your dictionary, and post your results on spdy-dev. That's how they got their change in the spec in the first place.
> ... but as protocol designers from Google the work done with Spdy is just unacceptably bad. Any actual examples? If you had any real basis for your arguments, I'd be more than willing to back you up on spdy-dev.
Take a look at headers served by https://www.facebook.com/ or any large site that uses TLS.
Also, you claim that Google is basing this protocol on assumptions and you imply that they do not understand the value of metrics. I strongly disagree with this sentiment, mostly because Google's propensity for data-driven experimentation once wasted an afternoon of mine.
I once spent an afternoon debugging a SPDY server that wasn't negotiating spdy/2 over NPN properly. Turns out that for 5% of startups Chrome will disable SPDY, fallback to plain HTTPS, and collect anonymized performance metrics. You will, of course, argue that this is not a fair comparison with a pipelined HTTP stack. I have posted before about the issues with pipelining, and won't repeat myself here. Suffice it to say that pipelining has many problems; problems of a large enough magnitude that it might be easier for a browser to implement a new protocol than it would be to (correctly) apply the many heuristics necessary to enable pipelining in the wild. SPDY would also cause requests and responses to conform to an asynchronous model, which (to me) is wildly preferable to the synchronous one prescribed by HTTP pipelining (and HTTP in general).
(edit) And to answer your question about why \0\0\0\4 and other lengths appears so many times in the newest version of the prefix dictionary (the 2nd one AFAIK), its because SPDY headers are length-prefixed for ease of parsing. It compresses better when the prefix is included in the dictionary; such that {0x00,0x00,0x00,0x07,o,p,t,i,o,n,s,} would compress to one byte, not 5 (assuming that that entry was still in the dictionary of course).
Pipelining can only be done for idempotent requests, (ie GET only), has head of line blocking problems, and can break proxies.
Also, its very difficult to handle errors in a pipelined situation. For example, if the 2nd request in a pipeline fails, but the server has been processing 3rd and 4th concurrently, what response does it give?
I think most engineers would agree that multiplexing over a single connection is better and less error-prone than pipelining.