My biggest issue was with server rendered applications. I think it's a good fit for SPA's where each piece is rendered to the browser independently.
Just as an example, my go to test for most programming languages is to rebuild my personal site with it (http://brightball.com/). Just provides me with a good exercise in running through a simple project that I've done in multiple languages and provides a good basis for comparison without getting too fancy.
If you'll look at the page layout, you'll see the main content area with a list of articles, some sidebar elements that are loaded from the database and a list of primary tags in the footer (also pulled from the database).
The biggest pain point was the reusable elements (sidebar and footer) with the template system. In most other template systems it's pretty trivial to write parts that have code attached to them and tell a layout to call those pieces with some arguments. With Go, that was a bit of a pain requiring all of the data that that would appear on the page to be retrieved before starting the rendering process. That led to a lot of repetition to pull in that data on every path to the layout.
So the layout/partial system would be my biggest pain point. My first instinct was to go and reach for a framework but so much stuff that I read on Go boards gave the impression that the community seemed to have a very negative impression there.
Additionally, there wasn't a clear winner among frameworks which made me hesitate. That's a problem you see in languages that have so much stuff for the web built in like PHP and Javascript. It becomes so easy to do-it-yourself on the framework front that there are a million different flavors and no clear path.
With Ruby, Python, Elixir, Groovy, etc there are clear and dominant frameworks for the language. If you learn this language, you should know framework "X". When there isn't that clear winner, it's hard to justify investing in developing around it.
To try to get around that issue and the sense that with Go you were supposed to sort of assemble your own parts, not commit to a framework, etc I tried https://github.com/runemadsen/ok-go since it's basically a framework of assembled and replaceable parts.
After a while I just gave up on it and started looking at porting to Hugo before I found out about Elixir and gave it a shot. Ultimately, that was exactly what I was looking for (http://brightball.com/articles/insanity-with-elixir-phoenix-...).
Most of the issues I describe are non-issues if Go is backing a React/Angular app instead. I just found it to be a huge pain for entirely server side applications.