As someone who works with Go full time I can understand where this criticism is coming from, but in my experience it becomes a non-issue after very short amount of time. The various editor helpers that automatically add/remove imports help.
Not using Go here, but I configure my Java IDE to report unused imports as errors; stil not a problem because it's probably been years since I last edited import statements manually -- days can go without I even reading imports because they're usually collapsed by the editor. Non-issue.
what editor and plug in you use?
The :Drop, :Import, :Gofmt and :Godoc statements that allow you to drop packages, import packages, cleanly format what you're working on, and document the targeted function are in the vim extras that come with Go, and work well enough to make this painless for me.
I don't know for Andrew, but I use Sublime Text 2. Cmd-., Cmd-P and type the import name, you're done (it toggles add/remove of the import). You don't have to leave your position in the file.