Video ram is still slow if you're accessing it from the CPU! Doing software 3D didn't really let you make use of whatever (limited) acceleration video adapters had at the time, which were mostly aimed at accelerated lines and moving rectangles around (you know, things that are useful if you're Windows). That meant the CPU was having to calculate everything and then push pixels over the bus.

This was why people were willing to use things like the VESA Local Bus, basically an extension of the 486 bus usually used to drive a graphics card and starting at a heady 100MB/sec. Terrible idea (it got faster as your CPU bus got faster, and if your card couldn't deal with that you'd have to underclock your CPU), but until graphics hardware started offering operations that matches what game developers wanted, the number of pixels you could push over that bus per second was genuinely a limiting factor.