Using container_of is better because it doesn't lead to a broken code when someone (accidentally) splices another field at the top of the struct.

Or put differently, direct casting comes with an assumption, while container_of doesn't. The fewer obscure assumptions are there in the code, the better. I think we can all agree on that :)