Hacker News Reader

Likely worth mentioning that Andy (the author) has been organizing an ML⇄DB Seminar Series (Machine Learning for Databases + Databases for Machine Learning) for the past few months (Fall 2023); materials & lectures: https://db.cs.cmu.edu/seminar2023/, https://www.youtube.com/playlist?list=PLSE8ODhjZXjYVdJKka5g3...

I actually just read a paper where some new file format achieved far faster and smaller writes than the Parquet used in all of these modern big datalakehouses, and it was done by guessing like 20 different simple compressions on a sample to figure out what will work best for each chunk of data. Sounds like manual feature detection to me!

Agreed this is an underexplored topic, which is funny because a few years ago prior to LLMs, if you asked someone how they'd apply ML to a database, you'd get an answer more like this than about any vector indexing or LLM shenanigans.