Your suggestion would kill GA (general aviation) if you had to fly a specific altitude for a given direction. A good gust and a rusty pilot could rise or fall a couple hundred feet pretty easily. Size of the plane matters a lot too. Plus, what altimeter setting are you using? (What is 2000 ft? AGL? MSL? Pressure altitude?) You would also frequently be required to violate these altitudes to maintain VFR conditions because clouds are moving into your airspace. And how do you do something like circling a point? How do you enter a traffic pattern? You would hit task saturation in no time.
I say this as a math lover (physics degree holding) and a pilot. Currently we have a rule; "Odd pilots fly East", so you could be 3500,5500,7500, etc. Going west at 2500,4500,6500, etc. Everyone should also be using flight following for anything but a really short flight. And flight following is really going to help with collision avoidance.