The difference with a quartz watch is that it's factory calibrated with the load capacitance on the crystal, and that it's a 32768Hz tuning fork. For a variety of reasons, generating higher frequency clocks off 32768Hz is... "annoying" (huge PLL ratio, very slow feedback loop step), and typical crystals in the 10-100MHz range are just less precise and thermally stable. (Not sure why, I'm not an oscillator manufacturer...)
(NB: you can of course correct for initial deviation in software. The actual problem is stability over temperature.)
Ed.: https://www.digikey.com/en/products/filter/crystals/171 (or, in the hopes the filter on the link works, https://www.digikey.com/en/products/filter/crystals/171?s=N4... ) - look at the options and prevalence for frequency stability & tolerance.
Ed.2: a wristwatch also benefits from being kept at constant-ish body temperature.