Rigidity of the frame is key for accurate and safe subtractive manufacturing.
You do not want your frame to flex while a spindle is cutting aluminium at 20kRPM. See Marco Reps and Kris Temmerman on youtube, they built formidable high precision CNC mills out of steel and concrete. For machines that rely on frames made out of extruded aluminium, flexing has always been an issue, even with much sturdier designs such as the Avid CNC machines.
Also, FYI, in a lathe the workpiece spins, and in a mill the tool spins.