Web Development in Go
kev.inburke.com
kev.inburke.com
Isn't picking a language to solve memory/latency issues a pre-optimization? Also I'm not sure if one can argue that libraries from one particular language are better at doing what you expect than libraries from another language.
Disclaimer: I'm trying to understand how much of what people are saying about Go is hype. People are telling me things like you can cross-compile on different platforms and output a binary, but I fail to see how these things are important for most API's and web apps.
That being said, it all depends on the use case. And about your point on binary output, its not just about being able to get one binary and running it in different platforms, its also about being able to run it without much setup in the future. For example, I had a hard time getting a nodejs app that I built a few years ago running again since so many libraries were deprecated and I had a lot of broken packages, but if it was built using Go, it would just be one binary with all the dependencies packaged in and I would be able to run it without any issues.
Personally I think it's fine to have optimization goals, particularly around things like performance, cost and security (or even your ability to hire engineers). Maybe you park some of them while you prove another point, but you can (and should) measure and adapt along the way.
Put another way, a high-performance platform provides a very high ceiling, allowing your application to be more brute-force with its implementation. You don't have to be sub-consciously worrying about performance gotchas while implementing your code because a high-performance platform gives you freedom to be inefficient within your application domain.
Not only that, a high-performance platform postpones your first scale event, and in some cases indefinitely defers it. I've found that applications on high-performance platforms tend to be easier to deal with on a day-to-day basis because they do not collect the various deployment performance-oriented cruft that you typically see with low-performance systems. High-performance platforms have simpler system architecture (fewer external processes, less monitoring overhead, etc.)
A bonus is that the user experience will be just that much better thanks to low-latency responses.
[1] https://www.techempower.com/blog/2016/02/10/think-about-perf...
However, the performance gap between Go and Rust is much smaller than the gap between Go and a dynamically-typed language, even one JIT'ed to the n'th degree, and the gap between Go and Java generally lost in the noise [1]. Between Go, Java, and Rust, you've probably got a good case that you'll get to more-or-less production-ready code fastest in Go. (And yes, "more-or-less" is a gap big enough to drive a truck through too. I preemptively pretty much agree with everything anybody might care to write about that issue in reply. :) )
[1]: I suppose I should point out that a perf gap of 2 or 3 is "generally lost in the noise". In the real world what tends to control which such language has the better implementation of something for most tasks is who has taken more time to write an optimized version. It's when you start getting up to the 10x-25x range that the gap between Go and a dynamic language can be that it's not really "lost in the noise" anymore; in that case, the faster language can end up simply faster than the slow language even if the slow language is given a lot of extra love (assuming no "change to a different language" shenanigans).
Making a platform selection that is informed, in part, on the language/platform's performance is a reasonable thing to do. As long as you can be equally productive (or even if you are a little less productive) with a high-performance platform, you will likely endure less pain during the initial several months once your system goes live and considerably less pain over the long haul.
Plus, it's really great to have a wicked-fast web app with very little performance-tuning effort.
You shrug off the established players but all else is very much not equal. Until Go has its own mature Django or WordPress projects, you're spending a lot more time reimplementing the wheel.
Yes, some platforms (e.g., WordPress) give you certain functionality off the shelf, but if your application isn't a CMS or blog, then the utility of that functionality is low. When building a custom REST or REST-like server for an application, a CMS platform isn't necessarily a reasonable starting point.
As with anything in technology, many variables need to be evaluated to make a decision. My point here is that performance is a consideration—more significant and valuable than many people understand—and it at least somewhat objective. Developer productivity is more difficult to objectively measure and varies by team on matters such as previous experience.
A framework gets you going real quick, but at some point, probably 3-6 months in, you'll need to do something that the framework doesn't like. At that point the framework suddenly becomes a hindrance rather than a help. You spend more and more time fighting the framework and writing more and more code that lives outside the framework. After a few years you've wasted way more time than you ever gained from using the framework.
Libraries are built to provide capabilities to your code. Until you've experienced a good standard library (like Golang's) then this makes no sense. It's common for newbies to Golang (like I did) to try and create supersets of the libraries and create their own framework. But every new project comes back to the simplicity and ease of using the standard library.
I like the OP's approach of having repos full of useful middleware and handlers. This makes much, much more sense for Golang development than a Django clone.
Try it, you'll like it.
It's great not to have to worry about frameworks, that often last just for a few years, than you have to rewrite or migrate to a newer partly incompatible version.
The fact Django is batteries-included, and everything is well documented in a single place, developers aren't wasting time working out what they can or should use for URL routing or handling database connections.
So no, I stick by my comment. A good framework saves time by being easier to learn, creating a more homogeneous product that is easier for other <framework>-developers to pick up, not forcing developers to form opinions about architecture they just need to work.
Yes it is, but if there are no perceived downsides then it's no biggie. I.e. Go is well supported by tooling, it has a good and growing open source ecosystem, it appears to support a level of day-to-day developer productivity comparable to other high-level languages, and so on.
In other words, the hidden corollary to the standard "premature optimization" meme is that optimization always has negative side effects (such as complex or difficult to maintain code). But sometimes, these negative side effects can be negligible and choosing a particular language over another is often one of those times.
For my project after a month in golang, I had to switch to C to avoid the GC pause. It also gave me total control over memory and made best use of CPU.
Will I choose C for every project ?, definitely not but based on what you are building, it is good to make that choice early on in the project.
Longer version:
- I've been writing almost exclusively Go code as my full time job for the last 2.5 years
- I've coded in Python and PHP full time before - each at least as long as Go (and neither of them was my first or second programming language)
- I was able to pick up Go significantly faster than either of the two (0 to push to production in less than 2 weeks)
- I'm at least as productive (measured in problems solved / features delivered in unit of time) in Go as I was in Python
- standard library is great and very often replaces the need for 3rd party libraries (performant, production ready HTTP server for example) and makes dependencies "flatter"
- deployment of a single static binary vs virtualenv hell (pre-docker) is bliss
- it's actually very fast for free - you'll start worrying about (and start spending time on) performance way later
Negatives: - dependency management is bad
From web app/api perspective:
- just using http server from standard library gives you concurency go is known for - each request is a goroutine which is cheap: minimal overhead
- you can delay worring about performance (at least from the app part - not db obviously) more. It's easier to throw hardware at the problem when you need 1/10 of the hardware
- deployment is easy - single static binary that just works
- multiple CPU cores are simply used: you don't have to run multiple processes (node)
- http server in standard library is production ready: no need to stick it behind something else
- you get all that without sacrificing productivity
Have you found a decent way to deal with the dependency management, I found it a bit clunky...
Previously, we had to ship virtual appliances to our customers which then involved debugging their wacky network setups, their wacky version of VMWare etc. etc. Now it's just one liner build, upload to s3, send out a link, done.
That depends on the outcome. Would you call using C++ for developing a videogame as premature? Or just picking the right platform up front to meet the goals?
Performance, especially efficiency, is way too often overlooked these days, and provides incredible benefits both for your own development, operations and end user experiences.
I'd say picking a platform is not, as it is definitely a critical part of your program, especially if it does not greatly increase the complexity of your code.
Theres nothing fancy in the syntax, you can pick up a project you put down months ago and easily start working on it again
Golang (and NodeJS) both feel really kludgy to me when getting into things like handling form submissions (validating input, redirecting with error/success messages, and all those tedious things that the popular frameworks abstract away the best they can).
Of course if you compare it with other options, you will find solutions that provide better expressiveness with less code (development speed), at expense of execution speed, type safety,...
I had to deal with large volumes of json (think log streams) that while they shared many top level attributes, also had had accreted many "optional" components with "types" that depended on the values of certain attributes (each with different nested elements too). While awkward to deal with in python (lots of `if x in y['thing']` checks), this was very cumbersome to deal with in Go. It required lots of reflection and ended up being comparatively slow (python json parsing is actually pretty zippy).
It just isn't one of Go's stronger suits in my opinion.
Maybe an opportunity for someone to write another library to better support more idiomatic usage of JSON.
I address this in the post; there's no problem with it, but it requires that you (or your end users) come up with a solution for deploying static assets to production, figuring out the relative paths between the working directory and the asset directory, and ensuring that the right version of the binary loads the right version of the assets. An example of the type of difficulty here is that you couldn't remember how to do it off the top of your head.
If you compile them into the binary, you (or your users) don't have these problems.
- there are middleware examples in the blog post and in the project
- depends on what you mean by "sucurity". there's an example of secure cookie encryption in the blog post and the project
- depends on what sort of authentication or authorization you need. If you just need basic auth you can try https://godoc.org/github.com/kevinburke/handlers#BasicAuth
I would go for new projects with something different than go. Like scala. Go is great for rest apis, but once your app becomes more "complete" in terms of requirements go becomes more and more painful in my experience.
You should probably write your own article then.
You can set the flash message in plain text, but I'd rather avoid the possibility of tampering, and it lets me reuse the same cookie code I use to encrypt e.g. session tokens.
If the key isn't sent to the client, how is the client supposed to display that message? Why are these messages encoded into a cookie in the first place? I don't think I understand the concept.
You also don't need to return every column if you know you only need an email address.
I know people that have had success with gorm or sqlx.
I really enjoyed working with both. It's nowhere near as magical as Rails, but it totally reduced the amount of code/boilerplate I needed to set up databases, migrations and to get Go "model" structs to and from the database.
We've summarized our whole experience with Go here: https://bugfender.com/one-year-using-go and we're just wrapping up another blog post with the Go pros and cons (from our point of view), which should be out soon as well.
The biggest problem with go is, that it is still a very fast and changing ecosystem. Which is great for a new language, but didn't serve us too well for building something stable.
It's the main reason i wouldn't use go for building a webapp that needs to discuss with a moderately large sql backend.
Asking for ORM means you are comfortable in your framework world. You will have to adopt, if you want to go a leaner way, with direct SQL or NoSQL API. Or find an ORM library (but learn about pro/cons).
All 3 languages have so much bundled in for web development out of the box that it's too easy to create frameworks...so there are lots of them without any type of rallying standard.
That's one of the reasons that I'm really averse to Go for web development that isn't just API based because you gain a lot from a complete, primary framework for web projects.
It's one of the things that makes Elixir so appealing because you've got a clear next-gen-web winner in Phoenix with a language that brings you so much of what's good about Go due to it's Erlang/BEAM underpinnings.
Since people do not unanimously use large batteries-included frameworks, you must realize that some people prefer some batteries a la carte.
Rails and Phoenix may win the large framework space, but they certainly don't win the web space.
Command line, workers, network and api layers are GREAT. If you have something that needs a custom solution...great. But the experience around managing a polished view layer, assets, routing, middleware, sessions, routing, redirects, migrations, database access, etc can get fragmented and tedious when roll-your-own becomes the defacto mode of operation.
When a language doesn't have all of the web-native stuff built in by default (like PHP, JS and Go) the community around the language tends to organize around a framework experience that will eventually lead to a common development experience that people can further develop tooling around.
It's not a knock on Go itself, it's just something that creates a noticeable side effect in the community in terms of fragmentation. People clearly WANT frameworks, because they keep rebuilding them. Doesn't mean you always need one, but it does mean that having a "winner" could be good for the community.
For example: https://vapor.codes
In particular Go is shaped by the experience of needing to run high-performance servers at a company with thousands of engineers; it was designed to address a lot of the problems with running C++ or Python in that environment.
Swift came from Apple, and was designed to address a lot of the problems with writing mobile apps in Objective-C.
It seems more like a compiler/language developer's idea of what a good modern language should be.
Swift was designed for writing apps on the mac platforms, primarily on iOS, and although somewhat debatable as a replacement for Objective-C due to it's quirks. If you want proof of that, just read between the lines in the original documentation.
Not to mention, many of the features that were called out originally are direct responses to issues with Objective-C: Optionals, Typed containers, Generics, modules, real first class functions instead of just blocks, final classes, let, etc.
Those things were obviously requirements for roles it should fulfill.
But no things in its design hold it back from being a modern language that's even more expressive and fast than Go. Except some concurrency love perhaps.
Where have you heard/read the part about Python? Not saying you're wrong, just haven't heard of it. I had heard about the C++ part. I know of some general issues (e.g. static typing, perf), of course, but interested to know which ones Google had (with Python).
Edit: The issues I've read about with C++ at Google (in Rob Pike's article) were, IIRC, C++'s complexity and long build times, even with distributed builds.
So the question is, what problem are you trying to solve? Does the language you choose address those problems effectively?
Are people writing their own web servers these days? is that a thing? Seems like everyone uses nginx.
“Your own web server” here means “your own program that serves HTTP requests”, and yes, people write programs like that all the time.
A web server is Apache, Nginx, Lighttpd, etc. What you just described is a Web Application, which I develop all the time on python -> uwsgi -> nginx stack.
For the record, I'm not blaming you, but the author for the confusing language. The title says "Web Development in Go" -- web dev is traditionally web applications/html/css etc, not typically including servers. Then their opening sentence is "You should write your next web SERVER in Go"
/pedant
That includes apache, nginx, etc. It also includes programs written in X language that process HTTP requests.