Some extensions are considered bad ideas later, e.g. -XImplicitParams.

Some extensions can make existing code fail to compile, e.g. -XOverloadedStrings.

Some extensions are fairly specialised and their use can cause error messages that are confusing if you don't know what the extension does, e.g. -XAllowAmbiguousTypes.

Some extensions can cause certain pathological pieces of code to suddenly start behaving weirdly, e.g. if you turn on -XApplicativeDo for a type whose Applicative instance isn't lawful, but whose Monad instance is.

Of course, many, many extensions could be enabled by default without making the UX worse or restricting the set of allowed programs at all: a large section of Haskell users is likely in favour of enabling syntactic conveniences like LambdaCase or MultiWayIf, or automatic code-generation utilities like DeriveFoldable, DeriveTraversable, and so on. That's what Haskell Prime is for, I suppose.