It a crime to enter an unlocked door, or even to walk into a building with no door at all, uninvited. This is a very different thing than visiting a public website.
Hackers tend to overemphasize the technical aspects of an issue, whereas courts mostly care about the human aspects.