Soon you will love pointers and structs and everything Go. Keep going (pun intended). Your colleagues are worried about their own work and your manager has made a long-term investment in you, not about the first few months. :)
I couldn't agree more that it is tough going when you realize a challenge is more than you expected. That plus impostor syndrome is what caused me to quit on my first try.
We are moving a lot of things from Python to Go at the moment and it has been great.