I've been exclusively using 64 bit computers and operating systems, both Windows and Linux, for about 7 years now, never had the reported driver problems, well, never had any issue, really.

For me at least, 32 bit is only important for ARM. On the other hand the issue is very much blown out of proportion, most people haven't seen it, even if they run 32 bit servers. Most usual servers written in Go, like web servers, use very little memory. I process 4k requests per second using 7MB of resident memory. There are many memory intensive applications, but you usually don't run those on 32 bit.