I’m confused. Are you advocating for complexity regardless of need?
I use parquet for small stuff now, because it’s standardised and so much nicer and more reliable to use. It’s faster. It ships a schema. There’s actual data type support. There’s even partitioning support if you need it. It’s quite literally better in every useful way.
Long winded way of saying: the complexity has always been there. CSV’s act as if it’s not there, parquet acknowledges it and gives you the tools to deal with it.
Once you've got the data into a processing pipeline you do need additional semantic structure to not lose integrity, which I think Parquet is fine for, but any SQL schema will also do.
CSV, being row-oriented, makes it much easier to append data than parquet, which is column-oriented.
CSV is also supported by decades of software, much of which either predates parquet or does not yet support parquet.
CSV, being a newline-delimited text-based format, is much better suited for inclusion in a git repository than parquet.
I use parquet wherever I can, but I still reach for csv whenever it makes sense.
CSV can not be supported by any software because it's not really a format. If you don't understand what I am saying by this, please write down an algorithm for figuring out which type of quoting is used in a given CSV file. Also please explain to me how you would deal with new lines in the text data.
Maybe you were being sarcastic, I'm not sure. But I hope it's obvious that advocating for complexity at all is not the same thing as advocating for complexity regardless of need.