Why I am betting on Julia (2014)
evanmiller.org
evanmiller.org
CL-USER> (defun f (x) (* x x))
F
CL-USER> (f 2.0)
4.0
CL-USER> (disassemble 'f)
; disassembly for F
; Size: 37 bytes. Origin: #x1002B10BDC
; BDC: 498B4C2460 MOV RCX, [R12+96] ; thread.binding-stack-pointer
; no-arg-parsing entry point
; BE1: 48894DF8 MOV [RBP-8], RCX
; BE5: 488BD3 MOV RDX, RBX
; BE8: 488BFB MOV RDI, RBX
; BEB: 41BBB0020020 MOV R11D, 536871600 ; GENERIC-*
; BF1: 41FFD3 CALL R11
; BF4: 488B5DF0 MOV RBX, [RBP-16]
; BF8: 488BE5 MOV RSP, RBP
; BFB: F8 CLC
; BFC: 5D POP RBP
; BFD: C3 RET
; BFE: 0F0B10 BREAK 16 ; Invalid argument count trap
NIL
*"Generic functions have appeared in several object systems in the past, notably CLOS [15] and Dylan [28]. Julia is distinguished from these in that it uses generic functions as its primary abstraction mechanism, putting it in the company of research languages like Diesel [10] and Cecil [9]." :-) ]
Behold!
CL-USER> (defun f (x) (declare (single-float x)) (* x x))
F
CL-USER> (compile 'f)
F
NIL
NIL
CL-USER> (disassemble 'f)
; disassembly for F
; Size: 37 bytes. Origin: #x10048EACEC
; CEC: 498B4C2460 MOV RCX, [R12+96] ; thread.binding-stack-pointer
; no-arg-parsing entry point
; CF1: 48894DF8 MOV [RBP-8], RCX
; CF5: 0F28D1 MOVAPS XMM2, XMM1
; CF8: F30F59D1 MULSS XMM2, XMM1
; CFC: 660F7ED2 MOVD EDX, XMM2
; D00: 48C1E220 SHL RDX, 32
; D04: 4883CA19 OR RDX, 25
; D08: 488BE5 MOV RSP, RBP
; D0B: F8 CLC
; D0C: 5D POP RBP
; D0D: C3 RET
; D0E: 0F0B10 BREAK 16 ; Invalid argument count trap
NILRead Henry Baker's paper on the CL-84 type inference engine. The intellectual ancestor of all modern dynamic PL optimization.
I'm not really famililar with CL, so forgive the perhaps dumb question: Does what you mentioned permit the user to write the completely generic algorithm while guaranteeing that it'll be specialized/monomorphized automatically by the comiler/runtime[1] when possible? As understood it, the thing the PP cared about whether it was guaranteed to occur and that it be automatic.
(I'm aware that these checks can be made very cheap, like JIT compilers do, but they're not zero-cost like compile-time monomorphization would be.)
[1] I mean if you want to avoid segfaults w/e.g. "No type checks".
Common Lisp is designed to be dynamic and the code that is produced should be able to work values of type T (anything) and dispatch it to the correct specific code. SBCL performs static analysis but the only useful way of doing static typing is to assume that known types won't change, otherwise you are going to widen types quickly and treat everything as type T.
The compiler can avoid doing redundant checks, in all modes. But it is allowed to blindly trust static analysis only if you allow it, because that's a dangerous thing to do. SBCL also has a special operator name TRULY-THE, a more aggressive version of THE (http://clhs.lisp.se/Body/s_the.htm), which declares that an expression is of some type U (e.g. (the single-float x)). Its use is reserved for cases where you really know what you are doing.
Is static typing as done here useless? Not at all, because sometimes you know that values cannot be redefined, for example inside a function.
I was writing a state machine, with local functions being used as states and a local variable that would be set to the function that represents current state. At one point, I compiled the code and I had a note about code removal being performed. The compiler guessed, based on the different `(setf state ...)` expression, the range of possible values for the state variable and determined that one state was not reachable. This is the kind of things for which the type system is useful, and dead-code elimination was safe to perform here because everything was done in the scope of a single function.
A generic `f` will probably be similar to the Common Lisp one. However, as soon as you use `f` in a context where types are known (e.g. if you use multimethods to overload a function that calls `f`), it will be specialized and JIT-compiled for the specific type (int, float, matrix, ...), with no extra work for the programmer.
* program compilation, where the object system can declare parts as non-extensible
* a runtime compiler, which can compile methods
See for example for a recent attempt to statically compile methods and handing the types to the optimizing compiler (which also uses type inference):
Racket is a beautiful thing.
CLISP also can use a JIT - it uses GNU Lightning for runtime native code generation (https://www.gnu.org/software/lightning/).
Anyway the Racket JIT seems to be a bit limited:
> The JIT compiler works incrementally as functions are applied, but the JIT compiler makes only limited use of run-time information when compiling procedures, since the code for a given module body or lambda abstraction is compiled only once. The JIT’s granularity of compilation is a single procedure body, not counting the bodies of any lexically nested procedures.
As the author says, the biggest drawback is libraries, and that is (1 of) the big attraction(s) towards languages.
Nobody wants to have to switch between C, Python, Assembly, etc. to make their code faster, but even fewer people want to write bindings to underlying C code. This also goes exactly against what the author doesn't want to do, which is to yak-shave around the language to get things working.
Python addresses a good deal of the problem with Cython, which lets you declare variables and compile to get ~90% of the speed gain you would get from a pure C implementation. And then you can just link to a C implementation if you still find you need to build one...
I think what's far more common is prototyping something in Python or R, and then realizing that 99% of its CPU time is already spent in wrapped C code. And then you call it a day and move on to other things :)
This is why Julia hasn't taken off nearly as quickly as I initially expected - Python/R/etc are highly optimized for their common use-cases.
Someone has to write those libraries.
> This is why Julia hasn't taken off nearly as quickly as I initially expected
Not sure what you were expecting, but consider:
- the most conservative userbase estimate I would believe is 20k users (based on mailing list subscriptions, website stats, and download numbers).
- sustained package ecosystem growth: http://pkg.julialang.org/pulse.html
Yes, but the discussion is about the "extremely common" case of prototyping, which definitely should not require library building, and usually should not require veering much from established libraries.
>Not sure what you were expecting
The data community can coalesce around a tool extremely quickly, in the matter of a year or two. Spark is about the same age as Julia, and has a thriving ecosystem around it. In 2004 R was a fairly esoteric analysis tool, but 2008-2010 it was the de facto data science language. Python made similar advancements in just a few years in the data science community.
Julia? I don't know a single person who uses it day-to-day, but I know a lot of people who tried very hard (myself included). The critical mass simply is not there: people aren't building packages because the users don't exist, and they don't exist because the packages don't exist. Your chart shows a linear growth in packages, which implies a constant amount of development work. This means it's not a growing language. This is what a growing language looks like: http://blog.revolutionanalytics.com/2010/01/r-package-growth...
This could be early exponential growth.
Or the constant growth may hit a tipping point as a critical mass of tools and infrastructure is developed.
Python is old and stuck with some serious design flaws which can not easily be fixed. The question is, can Python get fixed in shorter time than it takes to give Julia a competitive set of libraries.
For me, there is not something I can think of being such a deal breaker for me to switch to a another language. More likely it might not be ideal but could addressed by the ecosystem of Python itself.
Because Python has so many C/Fortran external dependencies.
Is Julia going to change it? By itself maybe, but currently it still borrows from Python in order to enrich its ecosystem, so basically adding even more layers. That for me, speaks to the volume that language is not really the pain here, library is.
Real-world problems are ugly deep down to its core and the requirements and constraints are every changing. Who thought GPU programming will be such a big deal just 5 years ago, if not DNN revolution completely turning thing around? And I don't think Julia's improvement is really prepared for this. Even more, if Tensorflow style data flow programming is better accepted, then probably any language could be used to just draw some the computation graph and the heavy lifting will be done by the framework anyway. It is going to be harder to persuade programmers to adapt a language just for the perks that is only peripheral.
This is a consequence of how easy it is to interface with languages like C in Julia. I.e. whenever a new CUDA library or change happens, it will take the Julia devs less effort to update their wrappers.
Add to this Julia's HPC support and it is by far the best offering if you want to do REPL style programming on a GPU cluster.
Once the wrappers are done, for most developers, it becomes the matter to layering those build blocks building their own functionality. It is not a strong enough argument that people should use Julia because it is a better language to write wrapper. Let alone that Python wrappers for data science stuff is already like a de-facto builtin.
Not saying Python is perfect, but IMHO, Julia is not hitting the right spot in order to really stand up against Python.
I must take issue with "near complete" - CUSOLVER.jl, my other CUDA wrapper, is missing a lot of the RF (refactorization) functionality and doesn't really have a pretty high level API yet. The doc and testing situation is also pretty bad. Everyone else's packages in JuliaGPU (the overarching GPU org on GitHub) are in a much better state.
You are right, though, that writing CUDA bindings in Julia is very easy - so easy that I can do it! It's also thanks to packages like Clang.jl, which make it easy for us to automate the procedure of wrapping the low-level interfaces.
Finally I must add the caveat that although I should test CUSPARSE.jl and CUSOLVER.jl with MPI (and GPUDirect) I've been extremely busy recently and not able to do so. If anyone wants to help out in this regard I would be very appreciative!
I've been using the remotecall functions rather than MPI (bunged my code into ClusterUtils.jl). Don't know how I'd use GPUDirect with this strategy.
The packages based on CUDArt.jl all seem to work with remotes. So long as you don't fetch a pointer to memory on a remote device.
Everything described here, for starters: https://news.ycombinator.com/item?id=11522767
(also: http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to...)
I love Python (despite the community). Sorry to be that guy but Python is always the second best and for me that usually means I end up using Python less and less. There isn't one area that Python is the best tool and sadly that is why it just isn't taking over the programming world.
Most of that was probably due to the slick GUI which nothing else comes close to (I've tried them all; trust me), but that is a legitimate factor.
Python has blaze, numba, dynd and dask going for it. These all ameliorate (and exceed Julia in some respects) many of the disadvantages of python (including fast user defined types). Then there are the libraries that while some can be used in Julia, you will never get full reliability, ease of use and compatibility.
On the other hand, Julia has amazing metaprogramming and much cleaner scientific syntax.
I think it will come down to whether Python can compile to fast LLVM regular standard lib programs.
Once (if?) Julia can be run in the browser with web assembly (using ahead of time compilation to produce small binaries), python has no chance if it doesn't follow suit. Python has no advantage that can permanently match Julia's potential ability to run on mobile, front end web, back end etc all from one beautiful codebase.
What does HN think?
That will never happen for Julia, because the language is designed to be fast for scientific computing... targeting web assembly is a pointless additional layer of abstraction that only negatively impacts performance.
I see value in Julia web notebooks, where the lifting is done by a server (or localhost), but what are the motivations for running Julia in a browser?
Interestingly, there's a slow-but-steady effort underway to implement a generic BLAS and LAPACK in Julia itself, since this allows for computations with custom datatypes that aren't supported by the Fortran libraries. For example, you can now do QR factorizations and solves with any numeric type, including Rationals and Quaterions: http://stackoverflow.com/questions/20985783/rational-matrix-...
Julia's dispatch system allows for these kinds of layered algorithms very nicely, and they could just as well live in separate external packages.
Can numba do this yet? Without having to write special "numba classes" code?
Blaze has a giant serialization wall between it and the JVM which it needs to solve to ever be competitive in its target market. Ibis is the much more compelling effort here.
Dynd is an overly complex C++ disaster that is trying way too hard to solve problems that aren't what anyone really struggles with. The number of real world problems for which homogeneous dense multidimensional arrays of floating point numbers is the solution isn't growing much. Real problems have interesting structure that your data structures need to be able to exploit on a problem specific basis. Doing the array layer all in templated C++ with Python bindings is not a recipe for making something easy to use and effective at problems it wasn't previously designed and compiled to solve. Displacing numpy is not going to happen in the Python space.
Dask has good ideas, but the insistence on implementing everything in pure python means you're stuck with the GIL and lousy performance of your scheduler. This needs to be low overhead if they want anyone to actually use it.
Dask distributed interfaces directly with HDFS...no need for serialization.
Dynd is specifically designed to deal with arrays of custom types...criticism sounds like it would be more aptly directly towards numpy.
once it gains Numba binding it will lose the Python overhead.
Re Dask...is 1ms per task too much? It has a threading scheduler that can work with numpy arrays to release the GIL....so not bound by that with numerical code.
Regarding regular list and text processing... Julia's poor performance with heterogenous data actually makes it slower than python for cleaning the corresponding dirty and text datasets.
Arrays of custom element types is a good first start, but boy is dynd a seriously overengineered way to accomplish that. I'm referring to array structure, sparsity, symmetry, linear algebraic properties that should be reflected in the type system. Python's type system is lousy for this, and C++ isn't extensible.
Julia can fix performance on heterogeneous data with gradual compiler improvements, and the string representation is due for a major rework. Python can't fix the fact that the language and libraries were not designed to be efficiently JITted, and extension interfaces are closely coupled to the CPython interpreter's API.
Revelation julia union etc performance...are these improvents a given, hypothetocal or hope? Doesnt fast code on these types fly kn the face of julia static optimization ethos?
Also serious question: dask's use of fast python datastructures like dictionaries gives it a 1ms per task overhead. Is that slow? How does it compare to other dag frameworks like julia etc
What does that even mean? Julia's union types aren't intentionally slow, they just aren't implemented very efficiently yet. Major revisions of how they're implemented are definitely on the roadmap, not far away.
For fine-grained parallelism of the type you'd use MPI for, 1ms overhead (assuming that's pure overhead above and beyond the actual cost of data movement) could be significant, sure. If you have calculations that need to go for thousands of individually cheap iterations, it adds up.
Well, luckily dask isn't meant for MPI style parallelism. I was wondering how it compared so something like computeframework.
Good to hear that union type performance will be optimized. Are there any issues I can follow?
Julia is designed for interactive, exploratory scientific programming, like Matlab and R. I don't see any reason someone would want to do that on a phone or in a web browser when it already runs natively on a PC. Then there's the matter of Julia being designed to interface efficiently with optimized native C/Fortran libraries.
As a former neuroscientist, I occasionally had connectivity analyses that would take 3 weeks to run. No way would I risk slowing it down with anything unnecessary. A 25% slowdown for using VMs in the browser would mean milliseconds for users and days for me.
While I agree with the author in "betting on Julia", I do not think inspecting LLVM IR or assembly of individual functions in the REPL is as useful as a feature as the author thinks, because it gives an inaccurate picture of what the actual output will be.
Peeking at the code at the level of functions means a non-inlined version of the function must be produced at the REPL, and anywhere else this function is used in your actual code, it will probably be inlined (the author's example certainly will) so it can be further optimized/eliminated by later passes. Furthermore, for a non-trivial function's dump, fundamental optimizations that increase code size (inlining, unrolling loops etc) would make the output confusing to compare with the Julia equivalents.
Thus, I imagine a lower optimization level is used to produce readable REPL dumps. For a performance minded person (the only type of person who would care about ASM/LLVM dumps at a REPL) even this is not helpful.
But beyond that, it generally displays exactly what gets executed. You can verify this yourself by looking at functions with different `-O` or `--inline=no` flags. And if you're interested how a function optimizes with inlining under typical conditions, you can just inspect the outer function.
That's why I'm betting on the strongly-typed functional tradition - OCaml/Haskell/etc. These language already have the killer feature from this article: they can be (almost) as expressive as Python and (almost) as performant as (naive) C++. But they also have really solid design with a lot of thought put into it that will let them evolve to meet the needs of the future.
I'm a bit mystified with respect to what passes as research in the field of type systems for dynamic languages.
https://stackoverflow.com/questions/17433435/what-makes-juli...
Im a sculptor, but all I make is coffee cups and ashtrays so I dont need to worry about the minor details of things..... This is what abstraction gets us.
Juila is one of those great things that everyone should use, but are simply bot promoted enough.
The main problem facing the language is maturity. That will come with time. Until it has fully stabilised, has a mature runtime, has a mature library ecosystem, and has strong debug and post-mortem tooling, it is not something everyone should use.
The author writes that Julia still isn't ready to replace his current development setup.
It is free/open source. And, it benefits from being a younger language and therefore has the hindsight of the quirks in Matlab's design. Coming from Python, the 1-indexed arrays are annoying but it's great for math.
I don't know how it is for others but for me I just into this mode where I think about Julia as doing math and I know in math you start at 1 for vectors and matrices.
Besides the way functions and APIs work, you seldom have to write code that really depends on the start index.
I wish I could get used to Julia but for some reason I just think the source looks..ugly. Maybe it's my dark past with php that makes me shudder at the sight of avg(x) (instead of x.avg).
I know that's stupid (and the syntax is actually required for multiple dispatch). Maybe Julia and I should try couples' therapy.
Since we are on Hacker News, maybe I should also spread a rumour. MathWorks, allegedly, in a recent settled lawsuit added a condition stating that the other company was not to create any Julia bindings for its product. If this is true, which I have many reasons to believe, it is a major recognition of how at least MathWorks sees Julia as a real threat to the Matlab market share.
I think Mathworks is feeling a lot of pressure from both Python & Julia. In the r2016a release of MATLAB, Mathworks essentially released a Jupyter (one of the first environments for Julia, and formerly iPython Notebooks) competitor with the 'Live Notebook'. Mathematica has had notebooks for quite a while, and the best Mathworks had was their MuPad suite. I think the prevalence of Jupyter definitely led to this development.
I would really like to not give Stephen Wolfram money every year, but evidently not quite as much as I really like to be free of paragraph-form monospace code blobs from hell. I was hoping that MathJax meant I could wait a few years and have my cake and eat it too, but so far I haven't seen anything that looks serious :(
- http://www.texmacs.org/ - http://www.lyx.org/ - http://maxima.sourceforge.net/
Since 1988, I believe.
I'd really like to have a replacement for cell arrays and struct arrays, that have some of the convenient features of those structures. I know you can get this with lists of dictionaries in python, for example, but it's not the same.