Tripping itself is easy to understand. This is from acid trip.

The balls part is like balls-out (a locomotive term, popular because it was similar to ballsy or they suspected it was related to testicles somehow) or balls-deep (an obviously sexual term). We like to make things about balls, IDK.

Tripping balls just means very very high. Typically mushrooms or other psychedelics, but not necessarily. You could come out of an anesthesia and have someone say “he was tripping balls when he came out”, and it wouldn’t be too out of place.