That's not much better than "everyone runs Windows" (but is a good way to view this). (And yes, you cover choosing an OS later in the post.)
What happens is that a select few browsers have what's considered at this point the runtime. So you are always leaving people out. Maybe the runtime price tag in itself isn't a problem now, but the hardware might be. The interface. And there will certainly be users who just want to avoid the low-contrast always-animating nature of some sites, for example.
Nowadays to me the environment is pretty much like Microsoft in the late 90s, early 2000s with their Windows+Office monopoly, and cultures that saw no problem is assuming that was available. But now, instead of these two, we're all expected to be running one of the select browsers that implement a recent enough version of the "living standards", and with powerful enough hardware and graphics acceleration.
But still, in some cases it has to be more complicated to make sites incompatible with other browsers and in the absence of javascript. Recently, I think I read something attributing trends in loading and rendering using javascript to the way Google ranks entries in search results.