DataFlow: Using LLMs to Build Reproducible, End-to-End Data Pipelines

We just released “DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI”, now featured as the #1 Paper of the Day on Hugging Face with early community support.

DataFlow addresses a real pain point in AI research and product development — reliable, reproducible, and scalable data pipelines powered by large language models. Rather than ad-hoc scripts, it provides:

A unified LLM-driven data preparation framework with modular operators and reusable pipelines.

Natural language to executable pipelines via automated planning and synthesis.

Strong empirical improvements across text, code, SQL, math reasoning, and agentic RAG tasks.

We hope this helps the community build better data workflows and improve downstream model performance. If you find this useful, please upvote on Hacker News and share your thoughts!