IMO, 20,000 is already way to much, but when that reaches 100,000 even just editing it will be a chore, let alone using it. It will simply be unmaintainable and unusable. I say this having worked on code bases which have reached that point and attempted to deal with the issues it creates. There's a reason every decently sized code base out there uses multiple files and a hierarchy. It's a natural way to create modular code, and modular code is simply better code.
That's why I still use a buggy BRIEF clone instead of a modern IDE. If I were forced to use someone else's idea of a code browser that was architected around files rather than subroutines, I'd probably be more inclined to agree with you.
The idea that the most appropriate way to organize source code happens to correspond to something that a (likely-long-dead) OS implementer chose to call a "file" is a notion that doesn't get challenged often enough.
On the same note, proper modularization of source files also encourages better discipline for dependencies among modules, such as discouraging circular dependencies.