Writing web applications using only the Go Standard Library | Hacker News Reader

Writing web applications using only the Go Standard Library

golang.org

296 pointsxorbox68 comments

I'm a big fan of this approach. Idiomatic Go often means doing things in an explicit manner. Consider how error checking works, for example.

I've tried a few of the Go web frameworks and it feels like they often add a lot of indirection. It's appropriate for Python or Ruby, but it feels out of place in Go. Yes you might cut out a little repetitious code but DRY doesn't seem to be a key pillar of the Go language

Go's standard library is excellent. I've found Gorilla Mux to nicely augment net/http and not feel like I'm wrapping magic around idiomatic Go. The only other 3rd party package I routinely use is sqlx.

http://www.gorillatoolkit.org/pkg/mux

https://github.com/jmoiron/sqlx

I literally found out about sqlx a few moments before coming here. I was stringing together ~10 columns by hand, and then when i fed them into the `row.Scan()` method, i just about rage quit. On Monday i'm going to try integrating sqlx and see how i like it.

Any other good sql libs i should know about? My coworkers mostly like sql by hand, so no magic ORMs preferably. But unmarshalling rows seems like an absolute requirement.. this is just anarchy without it.

Gorm. Give it a try. Light on magic. Can write by hand easily still. If you arent writing really complex queries and are just shuffling records around Gorm makes life a joy. It is no SQL alchemy but for most SQL users ORM/SRM makes life better. Anyway I think most systems using SQL in business logic end up implementing a relational mapper, just not as well as battle tested mappers. But some use cases do justify avoiding such tools.

Love mux, its super simple and still lets me use the standard library way of making web apps

Give Squalor (https://github.com/square/squalor) a shot.

You might find https://github.com/relops/sqlc really nice from an API design point of view (inspird by jOOQ, a Java sql library).

Sqlc is no longer under development, but you can easily use it as a reference to adapt other libraries to have a similar API.

I generally dislike the little things that beautify things. I don't like the decorators for routing in Flask, for example.

This is why I like CGI. I can make an effort to understand it.

Then you are absolutely going to hate the article I recently contributed to Gopher Academy's advent series, https://blog.gopheracademy.com/advent-2016/go-syntax-for-dsl... , about using Go to create terse, declarative DSLs for things like, say, building a webapp.

Here's a tribute to the tutorial as such, independent of the topic:

After having consumed thousands of tutorials, I found this one exceptionally lucid and instructive. Kudos to the author!

His approach of stepwise expansion and refinement lets him introduce feature after feature without the whole thing looking stilted or shopping-list cluttered. For each feature, he shares an itch with the reader and then scratches it. This keeps the reader interested and motivated.

Here, at least in my opinion, is a style and an approach that other tutorials would do well to emulate.

Look, I'm a big fan of go; I like the language, I (mostly) like the community, and I've seen some excellent stuff built in it.

...but, this 'dont use anything but the standard library' movement that keeps popping up here and on reddit is just ridiculous.

It's just fall out from people who can't be bothered figuring out how to use 3rd party dependencies.

No matter how amazing and excellent your language development team is, the man power of work in the wider ecosystem is significantly larger, and avoiding the work done by others to 'hand roll' your own solution is not the answer.

Yes; learning frameworks is tedious.

Yes... sometimes someone elses 'style' of code might not match yours...

...but, you have to ask yourself: are you writing code as an intellectual exercise, or trying to build something?

Because, if you're trying to build something, and you want to write every little piece of it yourself; you'll never build it.

This is the same (daft) argument as people suggesting they build their own game engines.

Building websites isn't trivial; and there are parts you need to do it aren't in the standard library, and never will be (eg. redis, memcache, elasticsearch, aws, grpc, the list is endless).

I appreciate the simplicity of just 'standard library only' to get started, realistically, is it meaningful to talk about?

I doubt you'll ever write a service that is 'standard library' only.

I mostly agree. Things you'll need in a more sophisticated app which are not in the stdlib and would be tedious to reinvent each time: named params, auth, csrf, nested templates, minified and fingerprinted css/js, grpc as you mention, wysiwyg editing, stats, oauth etc.

Many projects will only need a few of these, some will need all, and almost all non-trivial code needs third party libraries of some kind. Using go just for services and writing your ui in js just means you're shifting those problems into js libraries, and now you have 350,000 problems, including left-pad.

I agree this hostility to a 'framework' (a.k.a. a bunch of libraries) while containing some truth, has become a damaging religion. Library authors in Go tend to stress they are not writing a framework to appease the anti-framework gods, and new users are told to just use the stdlib without the caveat that larger projects will require more structure and pkgs from elsewhere.

The main reasons new users and in particular beginners look for a framework to get started with are quite simple - it provides guidance as to best practices, provides reassurance when they face unfamiliar problems, and gives them a community to ask questions in if they get stuck. None of these are unhealthy reasons.

I do understand and sympathise with the hostility in some quarters to frameworks as monolithic bags of code which everyone imports and uses when they don't have to, but feel this hostility is being cargo-culted on places like reddit rather than understood as an admonition not to unthinkingly import lots of code.

The "using only the Go Standard Library" was added to the title when it was posted to HN. The post builds an example app with the stdlib, but I don't see advocacy against using other code in it. I think python.org leaves writing a Django introduction to the Django people; this seems like that.

The Go community does have a tendency to avoid building or looking for all-inclusive frameworks, but, e.g., gokit.io, gorillatoolkit.org, and various third-party database tools (sqlx, gorm) are still popular.

This article is meant as an entry-level tutorial. And for that I quite appreciate how it shows the basic mechanics of the standard library. Of course, for a production application, you would introduce other libraries and frameworks as needed, but for teaching, it quite helps if not too many layers are added. You should add an external library to a project only after you clearly understood, which problem it solves and how :).

So, first of all, for the examples you cite - reds, memcache, elastic search, aws, etc. - I have never, ever seen a member of the Go community say "don't use a library".

The Go community is anti-framework, not anti-library.

And what's the problem with frameworks? Well, remember up until now most Go shops (including my own), are micro-service orientated. You want small, lightweight, high-performance services. Why would you want or need a large monolithic bag of tricks in there? If you need complex routing, are you sure it's a micro-service? If you can't keep track of the datastore and need an ORM, are you sure it's a micro-service?

Also, it's a discipline, not a requirement. In the same way other languages have their idioms (I'm a Ruby dev of 10+ years experience too - those idioms are baked in _hard_ for me), Go's idioms are around structure, performance, and what is reasonable to write at a larger level.

Should you use libraries in Go? Yes. Should you use frameworks? Nobody is stopping you, but are you sure Go is the right place for you and your monolithic application?

Also, I don't see a problem with the official Go website having a tutorial in Go using the standard library. Why would they not?

There's multiple good reasons for using a framework.

1) It enforces appropriate design patterns for the task at hand.

2) It produces code structures that are standardized rather than idiomatic.

3) It provides a superstructure that allows the integration of a variety of different plugins without having to write a lot of boilerplate.

I've no doubt this applies to go as equally as it applies to any other language.

That said, bad frameworks (which is most of them) are often worse than none at all.

I don't think you read my comment, or at least if you did, you didn't understand the point I was making.

Micro-services are the appropriate design pattern that means frameworks are redundant in a micro-service architecture.

Code structures do not need to be standardised in a service you can read the entire source code to in 20 minutes and replace in a few days if you need to.

Micro-services do not want or need the integration of a variety of plugins, and little to no boilerplate is required.

> ...but, this 'dont use anything but the standard library' movement that keeps popping up here and on reddit is just ridiculous. It's just fall out from people who can't be bothered figuring out how to use 3rd party dependencies.

Unlikely, I myself slowly came to arrive at this philosophy after 15 years in programming. Nothing lasts forever except bit-rot! Look if I want to look back at my youth in old age, that'll mean looking at, running, playing with old code.

Do I want my time spent discovering and documenting obscure bugs in rare contexts/situations of other people's code, or my own? Guess which.

Has a 3rd-party framework/lib been fine-tuned and painfully optimized for many months for my actual use-case, or their authors'/audience's? Guess which.

Is the stuff these actually offer any actual rocket science, or just pretty basic easily grokked stuff except 80% of it I don't even need and that 10% I really do and nobody covers turns out to be nigh on impossible to integrate with their very peculiarly own idiomatic ways of describing things which I wouldn't ever have any trouble parsing through had they used my own peculiarities? Guess.

Now don't get me wrong, am I going to reinvent CUDA, OpenGL, .NET/JVM or other crucial infrastructure layers, of course not.

But most helper libraries and bootstrapping frameworks are at best fluff for a MVP, then get that timebomb the hell out of your codebase built-to-last-the-ages. Do it for your golden years.

(I make an exception for 3rd-party Haskell code as of now. First, you usually just need functions together with the insights that drove their design and workings, not as copy-paste but to adopt-adapt and thus most efficiently learn from. Secondly, the core language is just 6 primitives or so, everything else in Haskell is technically syntactic sugar and language extensions, so writing "pure" Haskell would probably be as nightmarish as coding in Lisp s-expressions (then again, I sure know some delight in this). Thirdly, with equational reasoning and referential integrity guaranteed for every piece of Haskell code that compiles, the fragility is a lot less and mostly comes from IO interactions with the outside/real world, which I'm ultimately going to recoup control over myself anyway. Further, most Haskellers are as of now way more seasoned and I can only learn from their work. Even furthermore, even much/most of the built-ins are written for learners or clarity, not efficiency/robustness/all-edge-cases-correctness/etc and will need custom replacements with time, on a case-to-case basis. Lastly, it'll be a breeze (in comparison) to just keep various versions of older compiler versions around, it's all quite self-contained for the most part, Stack excels at this but one could decidedly do so manually in case Stack gets stuck decades later.)

I agree, this realization comes over time: when you have lived through a few languages and a few dozen frameworks.

The problem with frameworks is that they often turn out to be cancerous: one type of idea metastasises all over your code and then forces you to express everything in a philosophy that might get in the way over time.

In my experience: this eagerness to get entangled in frameworks tends to be inversely proportional to experience.

This is extremely, extremely bad advice. Do NOT run code like this, ever, exposed to anything, not even your own internal network. This is just ridiculously risky.

This gets you code generating html pages. It doesn't store anything. It doesn't do anything. It doesn't create any relation between any data and what is on screen. It stores things extremely insecurely (it can probably overwrite the application serving this with random binary data, trivial to privilege escalate (just overwrite the application itself)). It's slow, no caching layer, no ... It can't work, really, without tons of extra code. And it doesn't support all sorts of necessities of today's web apps. No auth, no users, no client identification, not even bloody cookies (although yes, all of those are doable with extra code). There's nothing in front of the application, allowing you to serve multiple applications.

This does:

    filename := title + ".txt"
    body, err := ioutil.ReadFile(filename)

(with filename taken directly from the url. It then proceeds to ignore any error upon loadPage return ... well done !)

(I love how the code making an extreme security error of using unchecked pathnames includes a discussion about exactly what permissions to use in the open call to open that file. Talk about missing the forest for the trees)

Just found a way to crash the application : just upload an invalid template, because the error is ignored on the template parsing, which can NULL the pointer the template, which will panic the app upon rendering the template. Hell, you're pretty much bound to do that by accident.

Please, people: DO NOT run code like this, unless you want to run a bitcoin miner for someone else, an illegal file mirror, or worse. For the love of God, do NOT give code on the same server any payment details for your web app and do NOT run this behind a firewall, or within your amazon virtual private cloud.

This brings back the very bad old days of C code serving webpages: so exploitable you're bound to exploit it by accident (ie. crash it). And it doesn't even have the good parts of the old C code : this thing uses introspection, which will make rendering templates about as fast as in python or ruby.

Just started with go ~5 days ago and I can confirm that after reading all the example/tutorial/gettingstarted posts this is a serious problem with standard golang. Error handling is seriously tedious.

can anyone else comment here on the stated un-safety of using this web app tutorial? I have no idea, but it seems like a good tutorial and it is posted on the official golang.org site. Can it be as bad as candiodari says?

Is this a new wiki page? I've seen bits and pieces here and there in the net/http docs but nothing nearly as extensive.

I think the stdlib is awesome, but the biggest gripe I have with the standard net/http library would be the multiplexer (ServeMux). Out of the box if you provide it with a handle like /foo but the user making a request adds a trailing slash (e.g. GET /foo/), it returns a 404, compared to other libraries like julienschmidt/httprouter which will handle cases like this for you.

For example: https://play.golang.org/p/Q96EulBUfI

Other than that I think the stdlib is bang-on for simple http servers. Wouldn't recommend it if you're planning on writing a web server backed by a database, however. Compared to other web frameworks like django, the stdlib is a great library for working at a lower level in the stack, whereas Django is more of a content-oriented framework built around higher level concepts like basic CRUD applications.

No, it has been around forever. It was the first page I used to see how to make a web app in Go.

The first thing people tend to do is to replace the default, very primitive "mux" with a router that more closely aligns with modern expectations, such as httprouter (https://github.com/julienschmidt/httprouter) or chi (https://github.com/pressly/chi).

These libraries also support middleware wrapping [1], a technique that the default mux supports but doesn't help you with. For example, adding CORS headers or logging the request.

[1] https://justinas.org/writing-http-middleware-in-go/

it's been around since before Go 1.0

I think Go is being strictly correct here:

https://play.golang.org/p/eSDl2MuFdP

Isn't being "technically" correct, but actually not doing what people expect kinda frowned upon?

> Be conservative in what you do, be liberal in what you accept from others

[1]: https://en.wikipedia.org/wiki/Robustness_principle

I despise that quote. Regardless of what the intended message was, what it implies is a horrible recipe for life-long support of shitty client behavior.

The good thing about Go's standard library is that Google uses it internally. Thus, it's well-exercised. It's probably been run on a bigger load than yours. That's valuable.

For me the biggest value they contributed with the Go standard library is the interfaces they baked into it, not necessarily the default implementations of them (no complaints there of course). They modeled the way that servers are built as interfaces, and if someone builds a 3rd party lib (like "mux", mentioned here in the comments), usually they implement the interfaces defined in the standard lib and they become a drop in replacement to the default implementation. I wish more SDKs were built like this.

"If your life is too easy and you want to create a never ending stream of tears and frustration you can try and inline the CSS that applies to elements above the fold and have the rest load in a separate CSS file."

Good read, not a ton of practical advice though.

I got a good laugh honestly. Nice article which is filled with interesting, although not necessarily useful performance ideas. Their site does load impressively fast at ~10ms page-load reported by Chrome.

Unfortunately, as a professional web developers, my clients never need such optimizations. They want new features long before they ever get upset about pages being sluggish in the second range. I'd love to be required to do more stuff like this, and I'm sure some people do, but the vast majority of the suggestions are looking for performance at all costs.

think you're commenting on the wrong article

I've written a tutorial on writing webapps without using a framework!

https://github.com/thewhitetulip/web-dev-golang-anti-textboo...

PDF: https://leanpub.com/antitextbookGo

I'm trying to make a similar for python, I knew it will not be as easy to start, Go standard libraries are more featured that the ones in python for the Web.

https://github.com/mariocesar/wiki.py

I've been working on a Go web server I hope to open-source (as soon as it's any good) and it occurred to me that one big advantage of using only the standard library would be in licensing. As long as your own license is the same as Go's (BSD IIRC) you present an unambiguous picture to anyone wanting to use the software.

In the end I think I'll use a few external packages and just expose the licenses in the app, but I'm doing this primarily for myself. If I were making something I expected to be in wide use I would seriously consider sticking to the standard library just for that clarity.

There was a discussion about Racket and its documentation yesterday. This is exactly the style of tutorial that they need.

Note, that this is not how anyone should actually write web applications in Go. Standard library doesn't have production ready defaults, in some cases even lacks options for that, and generally offers bad choices in terms of abstractions and performance. And as complexity of your app increases you are going to suffer the consequences.

Safer road for a webapp is to start from a net.TCPConn kind of server with your own tiny HTTP/1.0 parser and a tiny templating engine, your own or a 3rd party one (it's absolutely not hard). Pay attention to synchronization, packages that do implicit synchronization are better left to their owners, avoid unnecessary accidental complexity like that.

(Go user since Go 1.0, speaking from the experience)

Can you elaborate more on some of those production ready defaults the stdlib doesn't have? Genuinely curious.

By default there are no timeouts in net/http, for example. And you cannot specify them for everything. This has been going on for a long time, they fix one timeout here and there from version to version, as people report leaks, but to this day can't seem to cover all the bases.

https://blog.gopheracademy.com/advent-2016/exposing-go-on-th... covers a number of issues that should be considered.

That statement is incredibly hyperbolic, the defaults work fine for quite considerable scale (many millions of requests a day). Have used golang with http defaults in multiple high volume production deployments and rarely needed tuning.

> Safer road for a webapp is to start from a net.TCPConn kind of server with your own tiny HTTP/1.0 parser

Yet you complain Go lacks "production ready defaults"? Surely the built-in server would be more battle-tested here, and you can use its existing configurability, rather than having to enjoy the pain of implementing timeouts yourself after your server dies.

What does battle-tested even mean here. Not handling something as basic as timeouts properly for many years suggests that nobody actually uses built-in server for anything complex standalone, but more likely behind nginx and without direct communications with a hostile world. So it's more like not tested at all. But a decent abstraction over TCPConn with impossible to forget timeouts takes like 15 lines of code and is way more reliable, than using net/http.

Networking packages in Go's standard library are just too poor quality to argue about them. And it's not like they are different here, most languages don't have decent built-it networking libraries either, they usually are developed by 3rd parties as language matures.

Must admit I was bitten by the client timeout issue with my first real Go webapp. The thing was getting spidered here and there and would bail due to running out of FDs every couple of days.

> Not handling something as basic as timeouts properly for many years suggests that nobody actually uses built-in server for anything complex standalone, but more likely behind nginx and without direct communications with a hostile world.

Sure, but from what I know, Go actually supports them now. As problems are discovered, they will be dealt with. It saves you reinventing the wheel.

You had suggested writing your own simple HTTP/1.0 implementation. HTTP/1.0 is not enough for talking to the outside world. It lacks vhosts, it lacks keepalive. You lack HTTPS support too without adding your own TLS layer. You would have to use nginx or something in front for it to be ready for the outside world. By your logic, wouldn't Go's own HTTP server be fine, then? It'd certainly be easier.

Go's own http server behind nginx would be fine, until it won't. Which was my whole point. Let's say you have to deal with streaming through the server, you will have to learn the hard way that there is no way to set timeouts there even today.

"Yeah, yeah, but your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should.

Most, if not all, languages that work on the server side eventually develop a "minimalist" "web framework" in reaction to the "bloat" of some popular fully-feature one. Particularly coordinated languages even eventually develop a barebones server backend that allows you to potentially even plug them together, like Python's WSGI.

Generally these frameworks are met with general acclamation on HN. (Followed by a lot of replies generally defending the larger frameworks, which then devolve into a couple of threads about how hard it is to pack resources together, the importance of a "blessed" ORM, and the security implications of requiring people to assemble their own security stack for things like CSRF, etc.)

Are they less worthy if they're simply built into the language from day one? A lot of recent languages are taking such a tack, after all.

I am a bigger fan of the keep it simple approach, which is my way of saying I prefer to write web applications in the web technologies directly intended for the given task.

I suspect Go is a newer and better language than Java, but this soooo reminds me of Java devs trying to do everything web related in Java. Ohhhh the pain and torment (guessing, apathy, and crying about how horrible things like markup and JavaScript are because they simply aren't Java).

> Ohhhh the pain and torment (guessing, apathy, and crying about how horrible things like markup and JavaScript are because they simply aren't Java).

Almost everyone complains about JavaScript, and very few of the folks I hear complain about it wish it were more "like Java."

I've never heard anyone say they wish JavaScript were more like Java. Can you tell me more about why they say that?

I heard it a while back, from people wanting classes in JavaScript.

Yeah people wish alien language is more like their home planet language, so there isn't a new mental model to learn.

Or, once they have learned the alien language, they see legitimate shortcomings in it that their native language doesn't have.

True but I guess many OOers don't care to try to think prototypically. I did and I conclude that OO and prototypes suck equally so might as well use the prototypes

I wished I could have `for...of` loop to iterate array in javascript, so I can say goodbye to `for(var i=0;i<len;i++)`. Now it's supported by all browsers. But it's so late. You cannot use it unless you only support very recent browsers.

Not true, you can use typescript and have it along with stays-out-of-your-way static typing, or only test in modern browsers and cross compile later.

That's another language. You can prefer Java. You can prefer TypeScript. They are just not JavaScript. (Nowadays there are so many source-to-source compilers to JavaScript. Even Java-to-Javascript compiler.)

What additions that TypeScript has do you find offensive? And that's why I gave my second option, which allows you to use modern JavaScript natively (without the additional features TypeScript has, if so opposed) and cross-compile when you have to target older platforms.

I haven't heard anyone wishing it were more like Java although Google does have a Java to Javascript compiler that they use for some of their web applications. I'd guess those people in some way wish Javascript was more like Java or at least that they could write Java in places they'd otherwise be writing Javascript. It probably makes a lot more sense if your backend is also written in Java so you can share code between them.

I've never heard anyone wishing something would be more like java.

Or JS for that matter.

Closure and Typescript added type checking/safety features, ES6 adds classes, Typescript even has generics and interfaces. Javascript developers will eventually add most of Java's features but keep their conscience clear by never uttering the phrase "like Java".

Last I heard, Java didn't invent type checking.

> I am a bigger fan of the keep it simple approach, which is my way of saying I prefer to write web applications in the web technologies directly intended for the given task.

Web applications is the task that Go was directly intended for. It was intended for some other ones as well but Web applications was the main one.

Go is great for web applications that aren't CRUD! The web portions of IPFS were a joy to write in Go.

Reply on news.ycombinator.com