100%. The good part is, if you follow the tweet thread, you can see they are trying to improve things. Hopefully the effort can land and the rest products can follow.
43 karma · joined September 23, 2024
It's almost impossible to for LLM to tell all the invalid rows at once since the data cannot be fit into the context window. If we prompt the model to thoroughly do data cleaning, there will be many try-and-fail steps. This happens to me as a human, I clean some rows, expect my program to run with the data, only to find there are more malformed data. LLM cannot get it right for now, actually I see many cases that LLM fails because it wants to convert types (e.g. string to date).
Based on my experience, the best way is simply to skip the data cleaning step in the planning stage (you can provide feedback asking the tool to not do any steps).