It's not clear whether a parameter in a neural network maps cleanly onto a synapse in a biological brain.
I think it's becoming pretty clear that they don't. First, scientists uncovered many additional ways neurons interact with one another[1]. Second, it seems that individual neurons do way more computing than in the simplistic ANN models [2].