The physics jokes are probably the best part of the show, and they're sort of an easter egg for people who get them. That's the part of the show where the effort invested (expert consultant, etc.) really shines through.

In general, I find the show mildly to moderately funny. I'm certainly not offended by it, as a lot of people seem to be. I just take it for what it is. It's a mainstream show that happens to feature some insider humor, rather than the other way around. Characterizations of the show as "nerd blackface" seem misguided, IMO. Legitimately offensive "nerd blackface" would be, for example, the portrayal of Asian-American characters in '80s comedies like Revenge of the Nerds and Sixteen Candles.