You Don't Want to Use CSV Files

CSV files mean a lot of work and are expensive… couldn’t agree more. I had the distinct “pleasure” of working with the food plan system for a get of around 60,000 people. Every iteration of every plan option across every demographic of person was controlled by the biggest CSV you’ve seen. And it changed multiple times a year, requiring a series of review meetings among a small group of IT to review and update everything. It was absurd how that entire system ran off a CSV.

I remember not being able to read Pandas parquet files using for example Vaex (https://pypi.org/project/vaex/). And it is a bit disgenuine not to enlist all supported arguments of `pandas.read_parquet`'s `*kwargs`. But I agree that Parquet is already 100x better than CSV.

Someone is trying to sell something. Fool me twice, shame on me. CSV is a convenient format but omits schema/subschema. CSV data are what you make it. With so many implementations so might as well be in twisty tunnels each a little different.

I always thought of CSV as something you import with Excel. I've added export features to plenty of programs, but never really thought about any other possibility for the data besides a spreadsheet app.