Some Insights from a Julia Developer
stochasticlifestyle.com
stochasticlifestyle.com
* My deep knowledge and experience with Python
* My entire codebase
* The ability to work on projects with colleagues who don't also switch
* The certainty that when I leave my current job, someone will be able to pick up after me
* Zero-based indexing
(assuming we're not just arguing over arbitrary-array indexing to input predefined constants at the REPL/code level).
To access an array via index, you basically have to take the reference to the array, the index (i), and generally multiply (i) by the space set aside for each element (e).
i * e
If you've offset the array, you have to take the offset index (o), and call a mapping function (m) that maps (o) to (i), so the result can be multiplied by (e). Though it should be said there is another theoretical alternative: you can figure out a way to map (o) directly to (i * e) without the intermediate operation or mapping (o) to (i).
Where (i) and (o), and the relationship between them, is known at compile time, you can theoretically get the compiler to magic that operation away.
But where (o) is only known at run time, you have to first fetch/generate (o), apply (m) to map it back to (i), then multiply (i) by (e) before knowing how to access an array.
If you know fetch/generate (i) at run-time, you don't need the map operation (m), you just multiple (i) by (e) and there's your array reference. But otherwise there's an operation (m) sitting around whose cost must be paid.
You can't handwave it away via reliance on cpu parallel instructions or anything like that because its by definition a serial operation: you can't multiply by (e) until after you've got (i), you can't get (i) until you've applied (m), and you can't apply (m) until after you've fetched (o).
Now where the cost of (m) is sufficiently low relative to other operations, or where the number of array accesses are low, the two may appear sufficiently similar and offset array access might appear practically costless. But the point where we generally care about these things is because we're doing them a very very large number of times on a sufficiently large number of indexes that can't be known or optimised away ahead of time.
And it is worth pointing out that addition operations and things are remarkably cheap in general. But if you're accessing these arrays themselves in order to do addition operations or something similar on their contents, then it may still take up a relatively high proportional amount of time relative to the entirety of the program/operation.
Avoiding theoretical discussion on the possibility of functions (m) that map (o) directly to (i * e) while being equal or cheaper than (i * e) in cost, there is another possibility that might appear in practice...
If the original array access operations are not optimal, (or slowed down due to other issues, such as in-optimal caching or memory access, then it may be possible for both offsets and array access to be equal in practice. If indexing by both (i) and indexing by (o) are both seeing an additional step between the getting of the index and multiplying by element size, or if there is sufficient lag because the CPU isn't doing work optimally, then in profiling and practice, both would appear to be running to the same speed. But this wouldn't imply you've optimised offset indexes, it implies your regular indexes and operations aren'y optimised enough already....that's why it appears you can get something for free.
Julia has great interoperability between Python (checkout PyCall.jl) and many other languages. You don't need to abandon your old codebase.
With regards to your other points, a lot of that is subjective and hard to argue about, but for me Julia has drastically changed the way I approach writing scientific code and is not just an incremental improvement upon the status quo.
Take a look at the link below, comparing a simple sum in different languages (among them Python and Numpy). The power of Julia is that there is no privileged code. Your code will be as fast as the base library.
http://nbviewer.jupyter.org/github/alanedelman/18.337_2017/b...
The C function is of course naive. I'm pretty sure that a hand-unrolled loop would be faster.
What's liberating is that for the things which aren't covered by such packages, there's no huge penalty for just writing it yourself. In speed but in effort too -- it's quicker to write the loop than to figure out how to make np.einsum do what you have in mind.
I'm skeptical any GC'd language can approach C/Rust unless they have explicit mechanisms to do data layout for using the cache/prefetcher to the fullest degree.
C is the leftmost dot, Julia is just to the right.
If these benchmarks have resulted in all the languages focusing on a fast BLAS, I think that is fantastic.
1. Yes, some languages do not make it easy to install and link against a well-tuned BLAS. The benchmarks were very recently updated, and you can see the level of effort required to get everything installed and working properly. This was all done in the open on the discourse message board.
2. Yes, the benchmarks involve testing very specific language-level features like iteration, recursion, and IO. And they're testing the ability of specific languages to do these things. Yes, of course in Python you'd look towards a vectorized library call instead of a `for` loop, but then you're not testing Python itself anymore.
[1] https://alioth.debian.org/forum/message.php?msg_id=182495&gr...
[2] http://benchmarksgame.alioth.debian.org/play.html#languagex
[3] https://github.com/JuliaLang/julia/tree/master/test/perf/sho...
Like this guy did -- https://pybenchmarks.org/
Julia is designed to make it extremely easy to write high performance mathematical programs with ease. Rust is designed to make it easy to do systems programming. Both are high performance for the things they are designed to do. Just like I wouldn't do systems programming in Julia (although I am certain KenoFischer would), I wouldn't want to do mathematical computing in Rust.
C/Rust/C++ fit that very well so I wanted to understand how similar claims were made for a GC'd language. C# for instance can work with value types and was wondering is Julia has similar constructs.
Looking at the performance docs[2] this is pretty clear in that array types(which looks like how Julia does Matrices) get allocated on the heap. You can clearly see the performance impact in the docs(1.95s vs 0.08s) that this type of behavior has.
While you can preallocate(which the docs suggest and is the only path for GC'd languages) it's not an ideal solution. If your type is smaller than a single cache line, like in the example above, you've just flushed a whole cache line just to bring in that one pre-allocated value. You also run into the issue if you don't know exactly how many values you need upfront which leads to pooling. In that scenario your pool may be large enough that you're bouncing between cache lines on different pooled objects.
This is the type of thing that you really need control over if you want performance "as good as C". Anything less will be a compromise. There's also the whole class of zero-cost abstractions that can get from C++/Rust which leverage all the above to great effect. That lets you get things like nom[3] which gives you high level semantics + productivity while maintaining parity with C.
None of this is academic, these are all optimizations I've used on shipping products that went out to millions of users. In each case we had fixed hardware with a limited execution budget and the 5-20x improvements we made were critical to us shipping a product that people wanted to use.
[1] https://discourse.julialang.org/t/how-to-know-if-object-memo...
[2] https://docs.julialang.org/en/release-0.4/manual/performance...
[3] https://github.com/Geal/nom_benchmarks/tree/master/http / https://github.com/Geal/nom
Regarding preallocation, you tend to want to avoid dynamic memory allocation in high performance applications anyway, so whether you do that in C++ or in Julia, doesn't really make too much of a difference. It is true this is a little harder to control in GC'ed languages than in languages where you have to do memory management manually, but the effect is about the same. Julia provides tools to figure out where you're using dynamic memory allocation and those tools will certainly improve in the future.
One thing I think is under appreciated from the performance perspective though is how easily julia lets you express data layout transformation to take better advantage of the cache hierarchy. I touched on this a bit in my JuliaCon presentation [1].
Yes! Which is why it shouldn't take me digging through 3 different documents and still getting it partly wrong!
C/C++/Rust make this easy by annotating the type modifiers(&/*/Box/etc) with how the object lives in my runtime system. Rust even gets bonus points for giving me aliasing information as well(&mut vs &).
Other languages such as Java have that optimization too. But it is quite limited because it can only be applied to data whose size is known at compile-time. There is also the problem that if some data allocated in a procedure is passed to another procedure, it can't be allocated on the stack because it can escape.
A programmer working in a low-level language can be smart and choose the best locations (stack or heap) for each piece of data. A language that does not allow explicit stack access (like C does), can never make as optimal decisions.
One thing that was really common for us to do was to instance a weighted graph(something like this[1]) per-actor. This means that you might have 10-300 floating point values in a block indexed by the node they interact with. It was really common to see one, maybe two values change on a per-frame basis. With the constraint above I'm now copying 300 floating point values every time any node changes which would be brutal for performance. Or I'd take a potential cache miss each time I touched the array which could be even worse if it was a reference type.
To be clear, I'm not saying Julia isn't really good at what it does. My complaint stems from the fact that you can't claim performance good as/better than C without having all these tools at your disposal.
We haven't even gotten into things like restrict[2] where things like Rust's ownership model let you get it for free[3].
[1] https://docs.unrealengine.com/latest/INT/Engine/Rendering/Ma...
They're not. The semantics of value types and reference types are different in the presence of mutation. So if you want uniform object semantics in a language, then objects that can be implemented as values must be immutable. There are many languages that have kept these independent and bifurcated their type system instead (Java, C#), but it's been a source a great deal of pain and frustration (e.g. Java's `int` versus `Integer` awkwardness).
Fortunately, there's nothing you can do with mutation that you can't do just as well by modifying and replacing an immutable value in a mutable cell – the compiler implements them the same way. Wrap an immutable in a mutable `Ref` and voila, you've got something equivalent to a mutable value type without exposing completely incompatible semantics in the language.
I think C#'s overloaded term for "value-types" may have caused you to misinterpret the above. Let me be more clear, if a value lives on the stack or the heap is completely separable from if it is mutable or not, that's my objection.
> ... just as well by modifying and replacing an immutable value in a mutable cell
In terms of "correctness", sure. In terms of performance see my 300+ float block example above. I shouldn't have to copy, modify, copy when I can just mutate in-place(and be explicit about that rather than relying on language semantics).
Also I can seem to find any mention in the docs about Ref's semantics aside from a passing mention in the FFI section.
In my case, julia/llvm was smart enough to figure out that the read and write can be eliminated. Hence, the julia code that replaces an immutable with a copy where only few fields are changed generates the same @code_native as the obvious evil construction (figure out where the field is stored; unsafe_store! to the pointer).
But I guess this optimization is unreliable, or at least it is not well documented when this optimization is guaranteed to happen. So the situation is not optimal, but also not as catastrophic as you would have guessed without reading the generated native code.
This is the niche that Julia wants to occupy.
To be clear, I have no beef with Julia, I'm sure it's a fantastic language. I take issue with people thinking they can get the same level of performance without explicitly controlling their memory access patterns/allocations.
98% of developers will never need it in their careers but when you do there is no substitute.
Pauses is indeed a problem, which then requires you to manually tune the gc to your settings.
> reference counting takes more memory and slower than a well implemented gc
This is a ridiculous cliche at this point. It might be true if every memory allocation was reference counted, but in C++ (and julia) almost everything winds up on the stack. What doesn't wind of up on the stack is usually being dealt with using ownership and move semantics. The number of reference counted variables in my C++ programs is usually 0 unless they are being shared across threads. Not only that, but within a thread, move semantics means that the reference count doesn't need to be touched.
While there may be 'conventional wisdom' I have implemented non-trivial software in both C++11 and Julia, and optimizing memory allocations happens in both and Julia required more thought. Julia made data structures and general functions easier to write so it wasn't as if it was a net negative, but memory allocation wise I don't feel the garbage collector made things any easier.
Then on top of that you have the myth of the 'well implemented GC'. Java, C#, D, Go and Julia are all languages where this seems to be a constant struggle. After significant R+D some are there, but if you look at D and Julia, it is a constant user complaint.
As for cyclic data structures, I'm not sure why I would do that in the first place and I'm doubly unsure why I would do it with pointers and fragmented memory allocations.
Take the JVM, a good majority of ORM databases uses Sun.misc.unsafe to do manual native memory allocation. Not because it's faster(because it isn't faster than bumping the nursery pointer).
They do it so they can control where disparate datatypes live in memory so that as a cache line is read in the prefetcher is already pulling in the next cache line.
Things like this will get you a 10-50x performance increase, which in some cases you absolutely need it.
(As an aside, we also take this further with abstractions of 3x3 rotation matrices using a variety of internal representations such as quaternions in Rotations.jl, and allow composing coordinate transformations in CoordianteTransformations.jl, etc, etc).
Touching on the original authors post about the package ecosystem, I feel it can't be understated enough how easily different packages can compose and be used together in Julia. I like to say there is a product rule here where functionality/productivity grows as the square of the number of packages/features you can use together naturally (regardless of programming language). We can use static arrays, rotations, coordinate transformations, etc in combination with differential equation packages to simulate a system or with an optimization package to optimize a coordinate transformation for SLAM given certain measurements, and so-on, at full run-time speed but comparatively little user effort.
Sure would.
You can also easily verify this yourself by looking at assembly code dumps for JITed Julia functions. You can see that the amount of code would be comparable to that of C for many common scenarios.
As for cache and data layout. That GC'd language can't control data layout and cache usage is simply wrong. Best known example is probably Go. You can nest structs to create contiguous blocks of memory in Go, just like in C. You can also create arrays of structs as contiguous blocks of memory.
Just because Java and C# doesn't work that way, doesn't mean all GC'd languages are like that.
A struct of concrete types in Julia is just like a struct in C in the way it is packed in memory. An array of structs in Julia or a nesting of structs will have similar memory layout as in C.
So I know this sounds really odd, but Julia despite being a dynamic language thus allows more optimal memory layout than a statically typed languages such as Java.
What is frustrating with promoting Julia, is that it simply sounds way too good to be true, so people dismiss the language before they have actually read up on it. I advice you to read up a bit more on the details of how the language works. It will then become clear why it can pull off stuff which seems far too good to be true.
- libraries
- community
- tools
Are much more important.
Here's the standard pitch for Julia: C-like performance with R/Python/MATLAB-like expressivity.
This is undeniably true. For someone coming from R/Python/MATLAB, you will be able to program as easily in Julia, if not more easily and more cleanly, and get much better speed.
However, what I've come to realize now is that you can get something similar from Kotlin, Nim, and Crystal, but those are much more general-purpose, and often are more consistent in their performance. They don't have the same numerical libraries at the moment, but they have a much bigger use-case target, and for something like Kotlin, a lot more leverage from the community.
Rust is a lot lower-level, but as the numerical resources grow (and they inevitably will) and layers of abstraction are added, what you will probably get is something very similar to Julia in expressivity but more extensible downward, and probably more performant. Rust has a ton of room to grow, and has many good role models from C (e.g., Eigen).
This also isn't even touching on PyPy and things of that sort.
I like Julia and still reach for it when I'm starting a project. Often it's worked great, but other times I have to go back to R or Python for some library that calls C, because the Julia library, while usable in a way that pure R or Python isn't, is still not actually as fast as C. Sometimes "on the order of magnitude of C" is not actually good enough, and you need C. Also, if you poke around for benchmarks that aren't posted by the Julia developers themselves, you'll see different conclusions about the real-world performance of Julia, so there's that.
I guess I read articles like the linked post, and I end up feeling like they're misleading. Is Julia better for data analysis than R or Python? Yes, almost certainly if you were starting from scratch and not calling any products of outside languages. Is Julia better than R or Python + C as it is in the real world? Not sure, because you still have to call C anyway at some point if you're being honest. Is it better than Kotlin, Nim, or Rust? I'm even less sure. If Julia wants to maintain long-term competitiveness, it has to reach out in the opposite direction of those languages, from numerics -> general purpose computing, because they are inevitably moving from general purpose computing -> numerics, and they are bringing a lot with them.
e.g. Kotlin is being used for Android apps, which is worlds away from what MATLAB and R users do.
Stuff like the ODE library that Chris has, or JuMP (for mathematical programming) that Miles Lubin, Iain Dunning and Joey Huechette wrote, or a number of other packages are simply not available elsewhere. The Celeste project, for example, achieved 1.6 PetaFlop/sec of compute rate on half a million cores.
Programming languages are abstractions. Some abstractions are better for certain kind of problems. Many of the amazing libraries that are available in Julia would simply not be easy to do otherwise.
Think about it - if these were incremental advances and no serious community would ever form - then Chris would have never written his packages. Julia has seen over a million downloads. In my biased opinion, it is more than an incremental advance. I think the right approach is to understand where those advances are and apply it to the right kind of problems.
There is a balance to be struck between the two extremes of nothing new will ever get adopted, and everything should adopt anything new and shiny that comes out.
[1] https://arxiv.org/pdf/1508.01982.pdf
[2] http://jump.readthedocs.io/en/latest/installation.html#getti...
With all due respect, this is just false. I can construct a million dimensional linear program for a large network problem in a few seconds in python. Solving it with cplex can take minutes.
I have a Ph.D. in computational mathematics and do this for a living. I write lots of C, and still almost exclusively use Python or Lua for problem formulation because the performance just doesn't matter.
I'm certain there are some cases where it's true. One I've encountered is doing real-time solution of MILP for adaptive path planning. Here the problems are of moderate size and you want to reconstruct them from geometry data every 10ms or so. Here Python will indeed bite you in the problem formulation stage.
Chris' whole point is that the biggest benefits of switching to Julia will be felt by the folks that are developing the packages and libraries for others to use. He's advocating that the best way to get you to want to switch isn't incremental language-level features, but rather it's first-in-class domain-specific packages.
This is something that will take time, but Julia's language-level features are uniquely positioned to enable the development of such packages. Chris is extremely productive (and definitely an outlier), but in less than two years he managed to coordinate and build a first-in-class ecosystem for differential equations.
So he's not advocating for you to switch at all — he's advocating for folks to build the packages (like his) that will get you to want to switch.
> But for end users throwing together a 100 line script for a data analysis? I don't think that this crowd will actually see as much of a difference between other scripting languages if the packages in the other languages they are using are sufficiently performant (this isn't always true, but let's assume it is). To people who aren't "pros" in the language, it will probably look like it just has a different syntax. It will be a little faster than vectorized code in other languages if code is in type-stable functions, but most of the differences a user will notice will come from the mixture of features and performance of packages. Because of this, I am not sure if marketing the features of the language is actually the best way to approach the general audience. The general audience will be convinced Julia is worthwhile only by the package offering.
(For context, this was after several hundred words of compelling, detailed examples of Julia's language-level advantages).
EDIT: the author then goes on to claim
> that "lack of packages" isn't really a problem with the ecosystem: you can find a great package that does what you're looking for.
with several examples, pointing to poor discoverability, uniqueness, distribution and branding.
He also recommends more practical, use-case-specific how-to guides.
> but rather it's first-in-class domain-specific packages.
You'll note I didn't cite lack of libraries as a reason I'm reluctant to use Julia. I agree that a solid ecosystem is critical for adoption, but my point is that, for me, it's not sufficient to overcome the human/time/cost factors.
More importantly: I can imagine writing algorithmic improvements to it for my use case, which is where I hit a wall with python + scipy/numpy/numba.
I can't go and use my python skill to teach a scipy routine about some feature I need.
> As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
Julia has offset array indexing (for essentially no additional cost) which is an amazingly useful feature! See eg. [2] Think of this feature as blurring the distinction between data (accessing an array) and computation (calling a function). The fact is that arrays as they are used (contiguous memory location collection) often carry more information than being just a dumb list of arbitrary data, and it's very convenient to expose that in their interface.
[1]: http://www.paulgraham.com/avg.html [2]: https://julialang.org/blog/2017/04/offset-arrays
Now there is the awesome FFTViews.jl package which goes one better with wrap-around indexing: https://github.com/JuliaArrays/FFTViews.jl
As in, I don't see how its possible without stretching the meaning of "no additional cost". I'm familiar with displaced arrays and the like in Common Lisp, so i get that you can do offsets and things without using much additional memory, and i feel its a small jump from that to arbitrary indexing.
I mean, sure, if you're programming with constants or common patterns or things like that, a compiler could theoretically figure out and adjust the references itself. Or you can use macrology (I presume, since julia is so close to being lisp in matlab clothing, there's no issues with this in theory). Or you can come up with some kind of functional wrapper to adjust offset indexes back to the base case...
But if you're actually using run-time calculations using returned numbers from a calculation to index N-quadrillion times into various arrays and you're swapping between a standard and arbitrary/offset index measure, which is about the only time I can see you'd actually care about the relative efficiency of repeated arbitrary indexing, then it can't be a pre-compiled thing. The offsets between the actual numbers you're getting back and the actual index into the arrays have to be calculated in an additional step at run time.
Now its true that's likely to be only an addition or multiplication operation for each one, (and if it isn't, the run time application of another transformation just cements my point) but perhaps my notion of "essentially no additional cost" is more strict...
julia> using BenchmarkTools
using OffsetArrays
A = rand(1000)
O = OffsetArray(A, 0:999)
function naive_sum(A)
s = zero(eltype(A))
@unsafe for i in eachindex(A) # @unsafe will eventually be folded into @inbounds
s += A[i]
end
return s
end
naive_sum (generic function with 1 method)
julia> @btime naive_sum($A)
1.025 μs (0 allocations: 0 bytes)
514.0960505118594
julia> @btime naive_sum($O)
1.023 μs (0 allocations: 0 bytes)
514.0960505118594
loop:
vaddsd (%rcx), %xmm0, %xmm0
addq $8, %rcx
addq $-1, %rax
jne loop
I have my opinion, and that is that zero is the proper number to start enumerations on, not just in programming, but for everything.
Everyone starts counting objects at 1. Zero means an absence of objects to count or list. I count that there are 7 cars in the lot. The first one is 1. If there were 0 cars, I would not have any to enumerate over. If I make a list, I always number the first one as 1 (or A). If there was nothing on my list, I would have nothing (0) to number.
Word processors start list numbering at 1. Printers number the first page as 1. Kids are taught to count starting with one, using one finger. I feel like this is only an argument because we have zero-based indexing in mainstream programming languages, although maybe there are some mathematicians who think otherwise?
I don't take programming languages as a reference point. It's the other way around. I try to figure out what the most natural, principled, and elegant way would be, and I would base my programming language on that, if I were to design one.
> Zero means an absence of objects to count or list. I count that there are 7 cars in the lot. The first one is 1. If there were 0 cars, I would not have any to enumerate over. If I make a list, I always number the first one as 1 (or A). If there was nothing on my list, I would have nothing (0) to number.
We should distinguish between two uses of the natural numbers, I think:
0. _Ordinal numbers_, for assigning labels to things. "This is car 3." 1. _Cardinal numbers_, for saying how many things there are. "There are 7 cars."
The relationship between the two would be that the number of cars is the number that is next in line when you have called out the indices (0, 1, 2, 3, 4, 5, 6) of all the cars.
> Word processors start list numbering at 1. Printers number the first page as 1. Kids are taught to count starting with one, using one finger. I feel like this is only an argument because we have zero-based indexing in mainstream programming languages
What is the most common isn't what determines, for me, what is the most proper. I will play along and use 1-based indexing when I have too, but in my heart I know how I think it should _really_ be.
> maybe there are some mathematicians who think otherwise?
Yes, though starting numberings on 1 is still common (maybe the most common). Zero is at least recognised as a natural number nowadays. Historically, this wasn't the case. Apparently, Peano started on 1 in his axioms for the natural numbers [0], and zero, as far as I understand, hardly had the status of being a number at all during much of Greek and Roman antiquity [1].
[0] https://en.wikipedia.org/wiki/Peano_axioms#Formulation [1] https://en.wikipedia.org/wiki/0#Classical_antiquity
But who thinks that way? I just see 7 cars, not an indexed listing. And anyway, the 0 indexed car doesn't exist.
The question for me is what one _should_ do, not what is usually done.
Note: Although I'd side with EWD on this one (for now, at least), I don't intend to reference it as an "appeal to authority", but as an example of someone producing a reasonable, properly worded and thought-through argument on the topic, which I've not really seen since. I'd be very interested in what similar arguments would look like from the other perspective. If someone would like to convince me, feel free give me a link to some reasoning :)
Also: I personally find "it's always been like this" to be a terrible argument in either case. Those are the reason we weren't able to standardize for one system of units, for example. In the Swiss army, there's a saying that is used to explain the various idiotic decisions and the bogus explications thereof a soldier has to endure:
Ist so weil ist so, bleibt so weil war so.
Things are like that because they are like that.
All stays that way because it was that way.
The one exception to that is when switching between languages. Sometimes one forgets which language one is in and a syntax error will result. Like using the dot operator to concatenate strings in Python or leaving semicolons off the end of PHP statements.
This view of yours as applied to something new is as old as the woods, progress and shifts take place regardless of that. Small mercies actually but some are still writing and maintaining COBOL applications and earning extremely respectable salaries from it.
All you are really saying is you will not be an early adopter. That's OK, there will be others. Your non-adoption is unlikely to be a shattering loss for you or for Julia.
Regarding whether improvements are minor or significant that's a matter of opinion.
————
Julia, like Dylan and Lisp before it, is a Goldilocks language. Done by a bunch of Lisp hackers who seriously know what they're doing.
It is trying to span the entire spectrum of its target users' needs, from numerical inner loops to glue-language scripting to dynamic code generation and reflection. And it's doing a very credible job at it. Its designers have produced a language that seems to be a strict improvement on Dylan, which was itself excellent. Julia's multimethods are type-parametric. It ships with really good multi-language FFIs, green coroutines and integrated package management. Its codegen is LLVM-MCJIT, which is as good as it gets these days.
To my eyes, Julia is one of the brightest spots in the recent language-design landscape; it's working in a space that really needs good languages right now; and it's reviving a language-lineage that could really do with a renaissance. I'm excited for its future.
There have been some attempts to forego this assumption and make a language that can do everything you need. This has led to the creation of Lisp, Forth, and Smalltalk, among others. (If you've ever wondered about the fanaticism generally associated with these languages, this is a major part of it.) Julia is attempting to be such a language, and in my opinion does a pretty good job of it.
> PHP initially implemented its loops by fseek() in the source code
A version of this would be: how can good package development be as easy as possible, and how can package use be as easy as possible?
I haven’t done any serious work in Julia mainly because the python libraries are mature, good, and performant enough. I can’t speak for everyone, but for end users in science labs library support is perhaps the biggest consideration for language choice.
Many of the language flamewars we do, tend to skip over the eco-system.
Which is much more relevant that any language design issue.
Specially given that the decision of choosing a language is a consequence of working with a specific tool, unless one is open to face some hurdles.
You can download a package, type in a preset command on a preset thing, and 95% of the time (its R, so its only ever 95% of the time), you get back an answer/number.
And if an answer is what you're after, that's where the considerations stop.
Was there an edge case? Is R's answer really correct? Did it wallop something in your search path? Do you have your profile set up differently to the author? Do all the dependent packages clash and silently depend on how you load them and in what version you did so? Has R coerced something in the background, or pattern matched your typo'd variable to something it shouldn't? Somewhere in your code did you hit on one of the thousands of gotchas?
Who cares? It gave me an answer. I can give it to my boss or put it in a paper.
I personally am all for faster, safer, stricter (while still being dynamic) languages. And i've tried to explain to many users that languages that produce errors when you do something you shouldn't are not your enemies, they're your friends. But I see many people every day aren't living that philosophy, they'd rather an answer than the right answer or a robust answer, despite what they'll tell you in plain english.
With respect to Julia, someone like me might like it (well, if they got rid of the matlab syntax and just went back to the Lisp they copied and relabeled as 'Julia' :P).
But perhaps many users don't want a faster, more robust language. Perhaps, many of them are comfortable with a simple but wrong answer. And for that purpose, I can't see where Julia wins out relative to R.
As I said though, these are thoughts i have in my darker moments...
R users love R because of ease of use. R does sometimes ignores ugly corner cases in favor of that ease of use (though I'm skeptical this damages the validity of the answer in anything like 5% of cases), but that's a side effect.
sklearn and pandas are great, but you still simply have to be a programmer to use them, or at least much closer to a programmer than many statisticians want to be. Allowing non-programmers to do things like
data <- read.csv(file='blah.csv')
fit1 <- lm(outcome ~ var1 + var2, data)
anova(fit1)
summary(fit1)
This omission of most quoting, the magic inference of column names, etc etc all makes R much easier to use.
Free. Quick. Interactive. 4 lines. Done. What's the value proposition of Julia compared to that? Now you can argue that Julia will allow better type specification and specialisation, etc, etc. But that's not what's valued or used in your example.
Now, my mindset/domain has to engage with such, and i freak out at the implicit-ness/assumptions inherent in such code, especially with some of R's additional design choices.
In my mind, having code that crashes/errors out when something ambiguous/dangerous is encountered in your example a "good thing". But that's not what's valued in the above example.
And while Julia would allow someone to program in such a way as to specialise or raise these errors and concerns, what is valued is the ability to not think about them. And if you don't think about them, or even as a library designer, if you want to program in such a way that your user primarily doesn't want to think about them, then you're effectively very close to reinventing/reimplementing a new "R", with all its costs and benefits.
I'm just making this observation explicit, even though I actually do value the things the Julia crowd seems to value, but I'm not sure I'm in the majority, and I'm not sure you can push that down into the libraries without exposing concerns most users don't want to be exposed to.
so my 02 on julia's value prop: free. performant. scalable. capable. easy to use is there, but the others matter a bit more.
At the moment the bottleneck in our scientific computing and data analysis workflow is not waiting for code to run but rather quickly inplementing, evaluating, and iterating different models on datasets.
It was said that v1.0 was due in early/mid 2017, but it looks like it's still a ways off.
There is (or was) a window of opportunity for Julia to steal mindshare in academia from MATLAB and R, but it feels like Python is beating them to the punch. Of course, Python had a 21 year head start.
In terms of community I think that the economics community and private equity firms in particular seem to have picked Julia up enthusiastically.
Having said that, I suspect that it will be three years before Julia becomes mainstream, and then it will be a minority choice for five or six more - if it is the big success it deserves to be!
It's always this problem, if everybody is using R, it is hard to migrate to Julia. But in order to migrate to Julia you need people to stop using R and switch to Julia.
(I don't mean to imply that _you_ should be the one to do it - I'm genuinely curious)
I say most of the basic stuff is there, but a few things remain. What kinds of things are you looking for that Julia doesn't have?
Compared to other solutions I've used like python + numpy + pandas, Matlab, Mathcad, heh even Excel, Julia is a breath of fresh air. Fast, clean, powerful.
The problem was extending it to be more widely useful in my own research. I wanted to make those same compiled functions also work with complex numbers to integrate spectral discretizations of a stochastic PDE (instead of the finite difference one from before). I found some SO posts like:
https://stackoverflow.com/questions/30054019/complex-numbers... https://stackoverflow.com/questions/27906862/complex-valued-...
At that point it stopped looking like Python at all. I always found the SciPy syntax a little verbose since I had used a lot of MATLAB before (but I wanted to make this project not require a license to run) (this QuantEcon cheatsheet is a good demonstration of what syntax is like in my domain: https://cheatsheets.quantecon.org/). But to make this kind of "complex or not" logic work in compiled Cython, I resorted to conditional compilation (http://cython.readthedocs.io/en/latest/src/userguide/languag...). These days I understand that what I created was essentially a multiple dispatch mechanism.
Anyways, at around that time I started experimenting with other tools, especially Julia, because I really was getting frustrated whenever I had to "go beyond doubles" and write something that was extendable instead of a one-use script. Maybe there's some tricks I was missing, but I found it really hard in Python and MATLAB. Soon after, my PhD adviser and I started arguing about whether one of the properties in the simulation's solution was due to floating point errors. I couldn't convince him, so I wanted to write this integrator so it was fast and compiled, but allowed arbitrary precision so that way I could prove that it still existed even with very high precision. I couldn't find a page which explained how to do high precision arithmetic in Cython or Numba, so I completely gave up. Needless to say, I decided to re-write a small portion of this in Julia and it worked really well, pretty much instantly. Then I was digging around the Julia package listing and Viral pointed me to a big opportunity (https://github.com/JuliaDiffEq/ODE.jl/issues/64) and I have been developing a lot of Julia differential equation solvers ever since.
Obviously YMMV, but after computing a lot without a "first-choice language" (between R, Python, C, MATLAB, Mathematica) for quite awhile, I kept with Julia because I didn't have issues when I hit less standard tasks, and I found it very easy to contribute fixes to other people's projects because it was just Julia code. This stochastic PDE integrator story is just one (significant) project that led me in this direction.
When that happened to me, I wrote Sage (http://sagemath.org).
To answer the specific question you had about multiple dispatch: Cython's had fused types for a while now, although I guess they weren't so prominent in the docs when you were trying to solve your problem.
The other problem with your argument is, I think you have the dynamics slightly backwards. Complicated libraries will go where the users are, not the other way around. Maybe my NLP library spaCy would've been easier to write in Nim or D. This didn't matter. I wrote it in Cython because I saw that the userbase would in Python. This has proven correct. It also has the effect of drawing slightly more users to Python, continuing the feedback loop. But each library's marginal impact on the developer community is small, so you're never going to get the marginal package author to switch language for the sake of convenience. What good is it to have a convenient experience developing a package nobody will use?
It'll be a sad day if Julia starts to get such popularity that high quality libraries will be Julia-only.
Some languages, like the Algol ones, even have user defined indexing.
So it is not neither 0 or 1, rather whatever the min value of the index happens to be.
1-based 0-based 20-based
Generally, I don't think there is a good way to achieve flexible indexing without causing troubles somewhere, so I don't think Julia has really solved the problem.
"Non-standard" arrays with non-standard indexing already work in lots of packages. It could be better (that's one of the things that I am advocating for), but it's not a language tooling issue whenever it's a problem, it was the developer going `::Array` and thus requiring a contiguous 1-base index array where other AbstractArrays would actually work.
Julia has potential to become a great general-purpose programming language, but this indexing issue will practically limit it to the numerical computing community. Perhaps achieving that is already good enough.
You're not understanding generic types and its relation to (1). There's only one way to access an array: getindex. That's the function that's called with A[i]. However, you can use an immutable to put a thin (zero-cost) wrapper over an array, and define dispatches to getindex to do whatever you need it to do. So it's both implicit syntactically because the user just does A[i], but it's explicit because the user has to choose a different type. getindex is then usually inlined and then compiled according to the type, making it a thin abstraction over the implementation.
There are iterators which don't have a size or length. You can write generic algorithms which require an AbstractArray which HasLength and query at compile time for things like that and throw appropriate errors (those are called traits).
There is still a lot of development to do here, but the basics like this are pretty much solved except when new users treat Julia like MATLAB, but I'm not sure how anyone could control for that.
As evidence of this, check out JuliaArrays (https://github.com/JuliaArrays) which is a whole Github organization devoted to the development of alternative array types, like StaticArrays (which are stack-allocated immutable arrays) or CatViews (arrays which are non-contiguous and constructed from views of multiple different arrays). The nice thing about Julia is that, if packages are written to work with generic types, they can natively (and efficiently) work with these "non-standard" array types, making them easy to integrate into the scientific ecosystem.
-Stan Kelly-Bootle
In C, the array syntax is "mostly" just syntactic sugar for pointer arithmetic.
When you do "a[n]=value;" this is equivalent to "*(a+n) = value;". To get the nth cell of an array, you just add "n" to your base pointer "a".
Array indexing, therefore, is consistent with the pointer arithmetic.
That said, and funnily enough, Fortran, which is much older than C, has 1-based indexing (by default, but you can configure 0 based indexing, if I remember correctly).
https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...
[0] https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
Edit: Fixed link. Hacker News apparently doesn't understand the delimiting of a URL with "<" and ">". >-( https://tools.ietf.org/html/rfc3986#appendix-C
I don't buy it. Wouldn't people want their programs to run fast regardless of this?
* Python seems not so different, but once you add numpy then it involves a lot of ugly `np.arcsin(np.sqrt([...]))` type of things.
* R looks horrifyingly ugly but I haven't written anything
Coming from Mathematica 1-based is comforting, and it also matches the way people write mathematics on paper which is nice.
R not just looks ugly, but is quite horrible in the inside too. For example the variable scoping is one of the most insane I have ever seen.
But in general, I think having special purpose languages is mostly a bad idea. Firstly, having domain specific languages isolate different fields of science: If engineers use MATLAB and statisticians use R, the transfer of progress between these fields is hindered. Secondly, almost all special purpose languages for scientific stuff encourage horrible programming practices. Which typically leads to practice in the field lagging years or even decades behind scientific advances, because usable implementations are quite scarce, or tied to a domain-specific language.
But it seems R uses lexical scoping - that is, the y is captured at function definition time. That's extremely common and quite useful, in my opinion.
[0] http://andrewgelman.com/2014/01/29/stupid-r-tricks-random-sc...
The function f contains a free reference to a variable called a. This is satisfied in some sort of global environment.
Later, f is called from a function in which there is a local a bound to 100. Of course, this a which is local to that function is invisible to the free reference in f which continues to refers to the global a that contains 10.
It could be there is some problem in R, but whatever that is, this article isn't exposing it in a convincing way.
Same thing in Common Lisp (using CLISP):
[1]> (defun f (x) (* a x))
F
[2]> (setf a 10)
10
[3]> (f 3)
30
[4]> (defun g (y) (let ((a 100)) (f y)))
G
[5]> (g 3)
30
(It could produce 100 if the symbol a were marked for binding as a special variable; but it isn't. Thus it's just a free variable in f resolved in the global environment, and a lexical binding in g).
But if you're coming from something like Common Lisp (which is my point of reference), you soon discover that "Lexical Scoping" as implemented there and "Lexical Scoping" as implemented in R are two different things.
In R (paraphrasing) you generally have a search path of environments, and at run-time, R will find a variable by going from the inner-most to the outer-most environment looking for variables of that name. When it finds one, that's the variable R uses. What R does do is capture the path of environments that were in effect at the time of a function's creation. R has to do many of these variable lookups at runtime, which is also a factor that makes is so slow.
So what you get is a scoping scheme that, if i were being short, i'd call "actually more like dynamic scoping with environment search paths", and if i were being flippant "pretty much dynamic scoping dressed up to look like lexical scoping without any of the benefits/distinctions of either. If someone changes the applicable environments or variable later, R will just keep on humming along, pointing at new variable references that align with whatever variable name the original author of the code used, as long as that variable exists somewhere on the applicable search path"
It could be that R somehow gets some aspect of it wrong but you haven't so far presented evidence of that. (Maybe it's something you know a lot about and so it's obvious to you.)
y = 2
def mult(x):
return x * y
mult(6) # returns 12Domain Specific Languages for Statistics and "Math" have traditionally been 1 based. R is one based and S before that. The fact that Pandas went with 0 based was a huge disappointment for many. I use R.
I think it makes sense for enumeration in general to begin at 0. It's a mapping from the natural numbers after all, and zero is the most natural number!
What makes you think of programming iteration as a motivation for zero-based indexing?
1 Based
0 = 1
1 = 2
Zero based
0 = nothing
1 = 1
Seem important for mathematics.
print(users[users.age > 25].head(3))
Returns 3 elements and not 4!
Of course, I don't know what Pandas is like outside this particular example, or what Pandas even is. A database thing?
C strongly encourages you to think of arrays as simply address+offset pointers, so it absolutely should start with 0.
I think it would make sense to say it. The reason I don't is that people around me would misunderstand me if I did. (Around some people, it works, though!)
Yet I've never felt problems using 1-based languages whether Lua or Julia.
To me I just switch my mental mode to think I am doing math. I am used to mathematics using 1-indexing. And I am not a physicist or mathematician so it is not like I am steep in this tradition.
Of all things one might object to in a language, I don't grasp this carries so much weight for many people.
It's less surprising if you know there's still a version of the Julia REPL that works with sexpressions: https://youtu.be/dK3zRXhrFZY?t=5m58s
Because the compiler's available at run-time, queries can be compiled into efficient kernels that can run in parallel over multi-gigabyte arrays orders of magnitude faster than MySQL or Postgres on the same hardware.
Without Julia, I would have had to find some other way to compile queries into machine code at run-time, which for me would have ruled out all the languages you listed. In practical terms I would have abandoned the project.
>Without Julia, I would have had to find some other way to compile queries into machine code at run-time
Note that you could also do it in Common Lisp.
But I agree, Julia is very powerful, perhaps one of the most powerful recent languages.
Is it really unique? Isn't that just a Monoid (e.g. in Haskell, Scala, ...) or am I wrong?
Could someone please explain the difference between the two terms? I've always used them synonymously.
1. Developers. People interested in software development that just happens to be scientific in use. Basically anyone whose considered writing a package.
2. "The Fuzzy Middle Category". People who are pretty good at a language, know some software development concepts (testing, etc.) but whose primary focus isn't developing packages as much as it is answering questions, usually with a number of packages glued together with some interstitial code.
3. Users. "The steps I use to read a .csv file into R and then run a logistic regression are..."
They are, essentially, invoking software commands, rather than programming.
Now clearly, these three classes are blurry and people can move between them.
Julia's JIT is a simple plain method jit, the easy one. He doesn't describe the pro's and contra's of method jit vs tracing jit. In short, method jits explode in memory usage and forbid expensive optimizations. The advantages are of course as described easyness to work with, reproduce and debug. Most JITs start as simple method jit, and then advance to Tracing JITs. Esp. with performance orientated languages with a lot of vectorization potential.
It's a great language. But the JIT will be improved sooner or later. Esp. with the memory-expensive type-optimizations.
Is this something that works now, something planned, or just speculation?
- Writing code with Nullables is cumbersome and verbose compared to Swift.
- The object.method() notation is sometimes more readable, especially in more complex expression. Plus, in an IDE it works well with completion.
- The ordering of types in a file matters.
- Overall, Swift code looks a tiny bit more readable, cleaner.
- No interfaces.
But yeah I REALLY wish the Julia guys can come up with a nice way of dealing with Nullable.
But I actually don't miss the `object.method()` notation. I find it is so much more straightforward to compose things functionally when everything is a plain function. I feel methods just adds complexity to a language. For readability I tend to use the form `x |> f |> g |> h` if I need to make it easier to read a call like `h(g(f(x)))`.
I agree completion is a bit nicer with `object.method()` but function completion works quite well in the REPL and Juno. I use `methodswith()` to locate relevant functions for a type.
In some ways I think it is actually easier to deal with than in Swift, as Swift base classes have so many methods you can't find the stuff you are interested in quickly.
I have kind of wanted Swift to be the solution for everything, but I see that the API design philosophy for Swift makes it difficult to turn it into a language which is as nice for data science as Julia. There is no focus on making stuff like matrix classes, multidimensional arrays, shell integration etc.
Also I don't quite like that Swift ties me so much to an IDE. Writing Swift code with a plain text editor and a REPL isn't as nice as doing it with Julia.
This allows you to write fast code in julia, which is impossible in python (you can call into very fast C/Fortran libraries with nice bindings, though).
On the other hand, I really hate the syntax. One-based array indexing (ok, minor), blocks ending with "end", and most importantly unicode support.
Really, who thought that it was a good idea to allow symbol names that are unreachable on a standard US keyboard? WTF? This makes it supremely inconvenient to use libraries that happen to export symbols with crazy names. If you must, specify a name mangling scheme and let people who want to see crazy symbols use an IDE. Seriously, am I supposed to use a hex-editor when auditing julia code?
I totally agree with the author that package discovery and uniformity are a big problem. Indeed, this is even moving in the wrong direction: More and more functionality is removed from Base and put into external packages. I would prefer a more "batteries included" style, with uniform quality and documentation for standard functionality (splitting into different namespaces is good, though), python-style.
Second non-cosmetic problem is that the language documentation is atrocious, both from a completeness and pedagogical viewpoint. Pyhton is again the ideal to aspire to.
But the point of the article stands: It is easy to write fast julia code, and the Julia (non-C/C++) parts of the Julia core form a nice tutorial for how "good" julia code looks like. If a function is not well-documented, look at the source code. In python, you quickly run into the C-wall (function is really implemented in C and the way the wrappers work is really non-uniform); in julia this is more rare, and the wrappers tend to be much easier to pierce (ok, julia does a ccall, figure out the code of the target; the wrappers are human-generated and readable and don't come out of a complex build environment).
But yeah, you shouldn't use julia for systems programming until a good standard way for unmanaged code has been fixed, if ever (mixing types that are garbage-collected and types that are programmer-memory-managed, you want garbage-collected types for performance-irrelevant parts and programmer-managed types whenever precise memory alignment matters for your performance). Oh, and the nullpointer support sucks big time.
My personal hope is that the python->julia interface improves. Then python users will be able to profit from fast, easily developed julia packages.
Btw, the question should not be "is zero-based slower"; I know that julia has zero-cost abstractions. The question is whether is is faster, because of the avoided decrement instructions for the underlying pointer arithmetic: say, M=Matrix{Int64}(4,N), then @inbounds x=M[i,j] corresponds to x=unsafe_load(pointer(M), i + (j-1)*4). And the internal C code needs an extra decrement on i, because array data pointers use the C convention of pointing to the first element.
Re 4: This is a problem with discoverability and dependencies. Say, my code needs a heap-queue.
Sure, heaps exist, e.g. in datastructures.jl. But there is no canonical documentation / default choice in the base language, like e.g. heapq in python.
And if you want to figure out how to use custom orderings for your heap you end up reading the source code of all 2-3 heap implementations in datastructures.jl (which use different incompatible APIs), because the documentation fails at providing examples. Older julia versions came with heaps included.
Re 3: The REPL is not everything. By all means, define a global name mangling scheme that (with default options) symbol-names should be latex-displayed in the REPL or ijulia. I mean, when using a mono-spaced font you don't want to see a greek sigma in latex either! You want the code, i.e. \sigma, and you want someone (not yourself) to write a word-processor that inline-displays weird characters for your grandma.
Don't understand me wrong, I like julia; it is just that every single "mostly inconsequential" / matter-of-taste design decision pisses me off, whereas the really important parts are awesome.
Re 1: Maybe I haven't written enough fortran in my life, but both python-significant-whitespace and C/java-style curly braces make a lot of sense and are very easy to parse and highlight (for text editors). But sure, I understand that this is matter of taste, and very cosmetic.
Bonus problem: By default, multidimensional arrays access is such that the first index is fast for iterations. This is different from C layout, and means that A[i,j] corresponds to A[j][i] if you switch out Matrix and Vector{Vector}. I know that fortran did it that way, and I think this was a mistake in fortran already.
If you watch Stefan's 1.0 talk (https://www.youtube.com/watch?v=qHpaztMu_Uw) you'll see that one of the reasons we want 1.0 out is so that way this kind of non-breaking work can get priority. 1.0 is about doing all of the breaking changes to the language. 1.x releases, and package work, are about actually using it. It's known that DataStructures.jl needs work which is why it's even mentioned in the talk as a 1.x improvement goal.
Also I like that Julia prefers shorts words over special characters. C/C++ use far too many special characters.
That might run counter to my delight at unicode support. I think it is quite nice to be able to write mathematical code using the same symbols as used in mathematics. I use the Julia REPL to write this where you got latex completions to get unicode characters.
"Second non-cosmetic problem is that the language documentation is atrocious, both from a completeness and pedagogical viewpoint. Pyhton is again the ideal to aspire to."
I could not disagree more. Here is an account of Julia vs Python from mostly a pedagogical point of view: https://medium.com/@Jernfrost/python-vs-julia-observations-e...
A few points. Documentation is often easier to read. E.g. look at the example of the `print` function.
Function usually have more sensible names in Julia and you don't have to guess which package they are hidden in for common things such as working with strings, paths and arrays. Part of this is due to Julia multiple dispatch which allows reusing the same function name for related functionality, while python is forced to invent unique function names too often. Even when it doesn't make sense.
I'd say Julia adheres to the Python zen of least surprise. Checking if a collection is empty, can be done with `isempty()` which is similar to a number of other languages. Python in contrast treats empty lists as boolean objects which are false when empty. How is that obvious?
When writing latex, you don't want to encounter unicode greek sigma. You want \sigma, as 6 7bit-ascii chars. Same in julia; sure, define a display mode that translates certain things into unicode for people like you, but keep compatibility with code-editors/people/tools who do not understand unicode. In short: I want to it to be possible to use a dumb text-editor, not an IDE/word-processor. (ok, the text editor will need to understand UTF8, because unicode string literals are really important; but please never go beyond a small white-listed subset of 7bit-ascii for language tokens).
Not using UTF8 in this day and age is just begging for problems.
At least on my mac you got OS wide tools for working with unicode.