It's usually a satellite (or even virtual) location (or locations) that the parent university administers. Extension schools usually cater to past-college-age adults for continuing education, are often part time, and are usually commuter schools (and don't really have a live-in campus culture). I can't speak to any differences in quality of instruction, but I wouldn't be surprised if the admissions requirements at most extensions schools weren't as strict as for the parent university (which makes sense, because people attend for very different reasons, often at different stages of their lives, with different backgrounds than you'd expect for a high school grad going to an undergraduate university).

I think, though, that the disdain for saying "I went to Harvard" when you've attended the extension school is rooted more in emotion/ego/pretension than anything else. A "real" Harvard attendee might not actively look down on someone who attended HES, but would probably scoff at the idea that they "went to Harvard". Higher education has a lot of weird social signaling and other crap that's hard to unpack.

But I do think that the accepted meaning of "I went to X" is that you physically attended classes full-time at the university's main campus. Saying that, when you attended their extension school, is at best misleading.