Windows CE isn't in ROM, the binaries are loaded from GD-ROM disc for games that chose to use it. And booting it wouldn't do you much good anyway since there's no graphical or text shell. It's intended to launch straight into the game.
For games that didn't need to be cross platform, it was still less of a hassle to develop on Windows CE because you got all of the services that having an OS provides you instead of having to write to the bare metal, which is what the Sega SDK required. Many developers chose the Sega SDK anyway because bare metal was what they were used to but IIRC that generation was the last console generation where bare metal was an option for developers.
Game devs in general did have issues with multithreading and OS scheduling interfering with game logic until mid-late Vista era; games used to be all big single threaded main loops with some interrupts for peripherals that struggled to adapt to multi-core/multi-threaded CPUs. DC was between PS1 and PS2, so games were still quite deep in that phase.
DC not having the OS in the ROM is IMO a big shame considering that Dreamcast could have been basically the Xbox. Makes me wonder if Gates' push on "bringing back games to hands of Americans" was partially driven by that "incident", if it had been one, on top of having the market dominated by Nintendo and Sony.