Every now and then, it seems as if some referees are a bit unclear on the details as well. (It is admittedly a bit squishier than when there is a hard and fast line on some field or court.
It’s hard to get right though, which is different. Having both ref’d and played soccer, you absolutely need lineman watching the field closely.
Under IFAB law, you have to keep track of where the potentially offside player relative to other players is at the time that the ball is kicked elsewhere, which means having eyes focused in more than one place at once. Not an easy job.