By your logic, someone that successfully guesses a weak password and ssh's into someone else's server and takes all of the data they can find there is doing nothing wrong, as they are "merely observing what their own computer was doing and making some TCP requests".

Obviously, this is not how the law sees it.