That is surprising. If anything, I would expect the toslink signal to be superior as it is completely galvanically isolated. Is it possible that there is dirt affecting the signal, either in the sockets or on the plugs?
Another possibility? could be placebo