> Do you really intend to be editing a 2GB file?
I don't know about most people, but I deal with such files (usually JSON) on a weekly basis. It's surely one of the things that makes implementing text editors tricky.
I don't know about most people, but I deal with such files (usually JSON) on a weekly basis. It's surely one of the things that makes implementing text editors tricky.
I, too, have dealt with huge files. My primary large file use case for vim is as an interactive grep, opening files, then just performing iterative filters reducing the file to something "manageable". If the file is too big, I might resort to raw streaming filters to temp files before going all vim on it (even vim has issues with huge files -- huge files are issues all their own).
But it's more a matter of pragmatism as to how far one wants to take a pet project like this. There's a curve of diminishing return depending on what someone wants from such a thing.
A single RAM buffer is a bad idea, to be sure, but raw horsepower of modern systems and CPUs cover up a lot of bad ideas. Simply the idea that you COULD mmap or even malloc a 2G buffer to suck the file in, and it would WORK, is enough to make ones teeth itch. I remember long a ago an article from someone encountering an early workstation with 1G of RAM, and the worlds it opened up. We have come far from those days.
If the OP is interested in buffer mechanics, then starting with an API backed by a simple buffer goes a long way. If they're interested in screen painting, window management, etc, then the RAM buffer may be all they need. Otherwise, they can work on replacing their back end with all of the assorted structures folks have mentioned, see what they like best, as they all have tradeoffs.
And I regulary work with JSON files the size of some MB, which already puts some editors into a real struggle.
Out of curiosity, why do you have so big files? (A whole DB as a json file?) And what editor do you use for them?
More like a whole database table as a JSON file - I split the whole database export into tables using `jq` (an export from Firebase Realtime Database). I generally use sublime text to open them, it's quite happy with anything that will fit in RAM. For very large file it just shows a load bar while it loads it.
It was vital for exploring the shape of our data during a data migration to Postgres as our data was not very regular in shape and firebase's querying capabilities are pretty limited.