HNHacker News
TopNewBestAskShowJobs

ayu05

2 karma · joined September 14, 2023

submissionscomments
ayu05··on Show HN: ViewKit – Interactive dataset viewer in the browser, no upload
Just added that for parquet files
ayu05··on Show HN: ViewKit – Interactive dataset viewer in the browser, no upload
Hi all! I've been working on ML/Robotics research for a while and often work with HDF5, Parquet, and Zarr files. Personally, I love the myHDF5 viewer, but there's no good equivalent for Parquet and Zarr, and switching between different sites also gets annoying. So, I built a tool that provides a unified solution.

It currently supports viewing several formats, including HDF5, Parquet, Zarr, Arrow, JSONL, NumPy, TFRecord, etc. I'm hoping to add more features/formats depending on what people find useful!

It's free to use with no sign-up required.

Everything is loaded and parsed locally in your browser (WebAssembly + JS), so your data always remains on your machine. It's also built to remain responsive on big files via efficient reading, caching, and prefetching. Traversing through data files actually feels faster than existing solutions like myHDF5 with simple caching/prefetching strategies. It also supports some common data types that existing viewers don't support (e.g. float16, complex numbers for HDF5).

I'd appreciate any feedback (feel free to comment or send a message through the website). Looking forward to supporting additional features/file formats that the community finds useful!

ayu05··on ViewKit – An interactive tool to view dataset files
I'm an ML/Robotics researcher and constantly deal with HDF5, Parquet, and Zarr files. Personally, I love the myHDF5 viewer, but there's no good equivalent for Parquet and Zarr, and switching between different sites also gets really annoying. So, I built a tool that provides a unified solution: ViewKit

For now, it supports viewing HDF5, Parquet, and Zarr files, but I'm hoping to add more depending on what people find useful! Everything is loaded and parsed locally in your browser (WebAssembly + JS), so your data never leaves your machine. It's also built to remain responsive on big files via efficient reading, caching, and prefetching. From using it, traversing through data files actually seems faster than existing solutions like myHDF5. It also supports some common data types that existing viewers don't support (e.g. float16, complex numbers for HDF5).

It's free to use with no sign-up required. I'd love for people to try it out!

I'd appreciate any feedback (feel free to comment or send a message through the website). Looking forward to supporting additional features/file formats that the community finds useful!