1992 Cirrus Logic GD5402/AVGA2 (ISA Bus) allowed you to enable special 128KB linear window directly mapping A0000-Bffff to VGA memory. Only ISA VGA chip fully supporting linear framebuffer access was ATI Mach64. Since ISA bus is limited to 16MB address space you had a choice of either no more than 12MB of ram, or finding a motherboard supporting "Memory Hole At 15M-16M" Bios option. VLB and PCI graphic cards mostly didnt have this problem (some mapped itself at 64MB assuming nobody would be crazy enough to cram that much ram into a computer in 1993). It popped once again at the end of AGP life/start of PCIE when we ran out of 32bit memory, this is why fitting 4GB of ram often resulted in 3.5GB usable at most.

Of course this problem is with us even today. PCIE Resizable BAR is a brand spanking new feature, first proposed in 2008, enabling expanding directly mapped buffer from previous limit of 256MB to full capacity of VRAM. AMD tried to repackage it under different name (Smart Access Memory) and upsell as exclusive to only highend brand new parts, Intel new lol GPUs require it to work reasonably fast, but lock out support on non Intel platforms.

Part of that was used for option ROMs like hard drive controllers, the EGA/VGA BIOS, network adapters, and for EMS; other areas were just marked by IBM as "reserved", which might have scared people off. In early machines there probably wasn't much of anything there, but then video memory needs were still modest enough that A000-BFFF sufficed.

Probably a good thing, because then those 'holes' could be used for UMBs...

The C000 segment was used for the EGA/VGA extension ROM. I'm guessing that using D000-EFFF would be unnecessary (because of the planar addressing squeezing 256kB of video memory into a 64kB address space), inconvenient (because the addresses wouldn't be contiguous - EGA and VGA were designed to coexist with either CGA or monochrome adapters in B000-BFFF) and (for VGA) insufficient - you'd still not have enough to map the entire 256kB of VRAM linearly. I also expect that IBM's engineers didn't want to take up all the extension ROM space because then it wouldn't be possible to add EMS cards, network cards, and whatever else ended up being mapped there. Though 192kB of write-only video memory in that space would be an interesting design!

A lot of early hard disc controllers used segment C800 for BIOS extensions. Other adaptors could map into the Dxxx and Exxx ranges.