Hacker News Reader

How do you store your data set? And do you have any open source releases?

You can store your data either remotely or locally (see here on how https://docs.activeloop.ai/getting-started/creating-datasets...).

You can then visualize your datasets if their stored on our cloud, in AWS/GCP, or you can drag and drop your local dataset in Deep Lake format into our UI (https://docs.activeloop.ai/dataset-visualization)

We do, with version control, Python based dataloader and dataset format being open source! Please check out https://github.com/activeloopai/deeplake.

"HuggingFace datasets" is an open source Python package: https://github.com/huggingface/datasets/

And they also have ready-to-use scripts for A LOT of the usual datasets: https://huggingface.co/datasets

including LAION 400M and LAION 2B: https://huggingface.co/datasets/laion/laion2B-en

thanks for the context and links! I replied to your comment slightly above in one comment. :)

Reply on news.ycombinator.com