> try loading a 15GB CSV in Excel.
Or visualising it in r or pandas without meaningful subsampling.
Or visualising it in r or pandas without meaningful subsampling.
It allows you to use Altair in Python for visualising data, but does the computation in the backend using Arrow DataFusion. Not for 15GB perhaps, but cool nonetheless.
The aggregate data is around 1.5 million experimental results. MiniTab is too unwieldy and requires too much manual reformatting of the data sheets.
Is this something I should be looking at in R or project Jupyter? Does one make better visualizations than the other?
Having many data points you want to explore you are always going to be at the edges of what your hardware and software can produce.
The last really big datasets I worked with were for my thesis and I had to do subsampling to below 10% to get results within 10minutes or so and that was basically plotting midi recordings of piano performances, so nothing gigantic