> If you do a ([32]byte)(foo) cast, you risk a panic if the file/packet/whatever is not the size you expected (e.g. because it's actually attacker controlled)

Can you give an example of a situation where that is actually a concern? It doesn't really seem like a realistic threat model to me. Knowledge of the key is pretty much how these algorithms define attackers vs. defenders. If the attacker has the key that's gg.

There are lots of things in Go that can panic. Even in syntax, the conversion is very similar to an interface conversion, and those haven't been a problem for me in practice, partly because of good lint rules to force checking the "okay" boolean.