This feels natural now, but other languages inherited a decision dating back to B, when logical operators like && and|| didn't exist, and the bitwise forms were used in their stead.
With that having been said, it has exactly one downside in my opinion: when porting C code, you really have to be careful sometimes. C has 15 or so levels of operator precedence, splitting similar operators into their own precedence, whereas Go has only 5 and they're always left-associative. Particularly tricky C code may actually make use of this to reduce redundant parentheses in ways that may not be entirely obvious... (especially when combined with implicit casting.)
Whenever I do quick expression language parsers for any reason, I always follow Go's operator design. It's very easy to implement and other than not matching C behavior, has less footguns.