Hacker News Reader

Full thread

wodenokoto

The workflow I would have imagined for versioning data is:

1) Load original data from source into quilt

2) Do transformation

3) Commit transformations to quilt, with commit message

4) Run experiment

5) Do new transformations

6) Commit to quilt

7) Run experiment

Rinse and repeat.

Looking at the video and documentations, this is not emphasised at all, suggesting that edits to data should be saved as a new package.

View on HN

You can absolutely edit in place and that will go into `quilt log` for the package--as long as you are the package owner. Our docs were a bit confusing on this point. I just updated them: https://docs.quiltdata.com/edit-a-package.html

We definitely imagine saving edits as creating new versions of the same package. The most common pattern we've heard is adding more dataframes or files to a package as new results are generated. But, we can certainly imagine other transformations.

Reply on news.ycombinator.com