247 karma · joined February 14, 2016
This could be early exponential growth.
Or the constant growth may hit a tipping point as a critical mass of tools and infrastructure is developed.
Well, luckily dask isn't meant for MPI style parallelism. I was wondering how it compared so something like computeframework.
Good to hear that union type performance will be optimized. Are there any issues I can follow?
Revelation julia union etc performance...are these improvents a given, hypothetocal or hope? Doesnt fast code on these types fly kn the face of julia static optimization ethos?
Also serious question: dask's use of fast python datastructures like dictionaries gives it a 1ms per task overhead. Is that slow? How does it compare to other dag frameworks like julia etc
Dask distributed interfaces directly with HDFS...no need for serialization.
Dynd is specifically designed to deal with arrays of custom types...criticism sounds like it would be more aptly directly towards numpy.
once it gains Numba binding it will lose the Python overhead.
Re Dask...is 1ms per task too much? It has a threading scheduler that can work with numpy arrays to release the GIL....so not bound by that with numerical code.
Regarding regular list and text processing... Julia's poor performance with heterogenous data actually makes it slower than python for cleaning the corresponding dirty and text datasets.
Python has blaze, numba, dynd and dask going for it. These all ameliorate (and exceed Julia in some respects) many of the disadvantages of python (including fast user defined types). Then there are the libraries that while some can be used in Julia, you will never get full reliability, ease of use and compatibility.
On the other hand, Julia has amazing metaprogramming and much cleaner scientific syntax.
I think it will come down to whether Python can compile to fast LLVM regular standard lib programs.
Once (if?) Julia can be run in the browser with web assembly (using ahead of time compilation to produce small binaries), python has no chance if it doesn't follow suit. Python has no advantage that can permanently match Julia's potential ability to run on mobile, front end web, back end etc all from one beautiful codebase.
What does HN think?
Does multiple inheritance
Julia is a dynamic language that can compile to fast binary code. Once it can be run in the browser with web assembly, faster and nicer syntax than javascript, why would anyone use python?
Also see this:
How about this? https://github.com/shashi/ComputeFramework.jl
But I suspect it has the same problem.
Edit: There is also this https://github.com/JuliaDB/DataStreams.jl
I'm not sure how much space just cutting out the GC will save?