Julia is awesome, but..
danluu.com
danluu.com
I find bugs and unintuitive behavior a lot when using Julia. Often it requires me searching the mailing list or Stack Overflow to figure out "idiomatic" Julia code when this really should be in the documentation (who knew that a matrix operation A += B allocates new memory for A each time?) And I think global variables should just be removed altogether, as many problems as they cause. I commonly find myself thinking "There's no way that someone new to Julia would know you're supposed to do it this way", whereas with Python or even C, that's not the case.
But I think the language has a lot of promise, and the language design is really slick and well put together: powerful macros, a nice type system, multiple dispatch, clean code when you don't need performance but lots of additional decorations you can add when you do (devectorization, @parallel, @inbounds, @simd, type annotations).
Remember that Julia is only at dev version 0.4. The author of the article complains about breaking backwards compatibility on an almost daily basis by removing or changing things. At this early stage, I see absolutely nothing wrong with this. I'm tired of languages that have unintuitive and quirky (stupid) ways of doing things just in order to not break existing code. C++, for instance, is abhorrent to me. How many pages is the standard??
At the moment, Julia has a steep learning curve if you want high-performance code. But once you know what you're doing, it's easy to get it running at the level of C. Plus, features like viewing the AST, or the LLVM and native assembly representation of the code are handy for debugging.
Hi, I'm curious what your thinking is? Is it because global variables make testing hard? Is it because global variables are can not be optimized well? Is it because of bugs in julia related to global variables?
I would like to use a language that doesn't have static variables, global variables, or static functions to make writing tests easier, but that makes it really hard to interface with C and the JVM. I often need functions like write() and System.out.print.
Mainly this; it's incredibly easy to accidentally use global variables and swamp your whole program. Think about defining some constant "n" in the global space and forgetting to put "const" in front of it. Then any for loops that use "for i = 1:n" will reallocate n on every iteration. If it's a nested loop, you'll get huge accidental memory allocations (this particular problem [not mine] is one on Stack Overflow that took a while to debug).
It's the same reason garbage collection is used; it's not strictly necessary but it prevents a lot of accidental bugs and makes programs easier to reason about. I think it would be the same with removing global variables.
* Of course, you still want function names to be global, so perhaps define a namespace that you can dump immutable functions or constants into. Or if there's good reasons not to remove global variables, at least make them all const by default and add a "mutable" keyword (and maybe swap that in the REPL). This would at least remove most of the accidental bugs.
To be honest, I hope they continue to break stuff. There are still many areas where it feels that the syntax/language itself could be improved a lot (too many to list!). Some of the changes have a significant impact on the entire system (like multiple inheritance for abstract types). Most of the proposed discussion/changes I've seen improve the language in several ways, they make it more uniform, so I'm really looking forward to them!
The base lib is very minimal, and the quality of the contributed packages varies wildly. It's a new language... you might lament most of the same issues with Rust.
Being a long time lisper, for me Julia it's a lisp in disguise with an edge on performance. The compromises/choices done so far have a great sense of direction and balance. It's a good time to jump into the development before the language is set in stone and the warts cannot be changed anymore!
This is not rocket science. You can do it too. Go read http://graydon2.dreamwidth.org/1597.html written by Rust's original author.
But if you consider the entire ecosystem (see http://pkg.julialang.org), many packages that you might rely on, especially for simple operations, might break.
Most of the breaking changes are developed in branches and merged when completed. Again, overall the source management is also following the best practices in this sense.
I didn't really want to get into too much detail because for me most of the arguments made are ok, but it's an ongoing development effort. When new ideas are proposed, I want them to be considered, not dismissed as "breaking changes".
[0] http://words.steveklabnik.com/rusts-documentation-is-about-t...
Also the ability to easily search both the standard library and third-party libraries is probably going to be needed soon if nothing experimental is going to be available in the "standard track".
It is a gut wrenching feeling when code crashes in production and it is my compiler to blame. It seems so much more shameful to me then if a library or service of mine crashed, not that that isn't shameful as well.
Do I need to loosen up?
As I've used more modern languages with better type systems I've moved away from high test coverage and towards writing code that simply can't go wrong (or at least, that is exceedingly limited in the ways it can go wrong); http://spin.atomicobject.com/2014/12/09/typed-language-tdd-p... is a well-written explanation of something very close to my current approach. So I wouldn't put the emphasis on branch coverage or fuzz testing that you do. But I absolutely agree that you should have a very high level of confidence in a compiler before you ask people to use it, substantially higher than the level of confidence required of a library or service.
https://groups.google.com/forum/#!topic/julia-users/GyH8nhEx...
My response (which is too long to include here) is this:
https://groups.google.com/d/msg/julia-users/GyH8nhExY9I/0Bzn...
> You cannot compare Julia with a project that has Google in the background. Its clear that they have a "more clear" development model and more documentation. Some goes for Rust. Julia is from and for researchers.
> Julia is from and for researchers.
I think the actual criteria is being ["a greedy, unreasonable, demanding programmer"](http://julialang.org/blog/2012/02/why-we-created-julia/) i.e. everyone.
Julia's strength so far is fast numerical algorithms, and that's what most people seem to use it for.
If installation issues are daunting, you may find juliabox.org a gentler way of getting your colleagues started.
See http://pkg.julialang.org/pulse.html, for example. We have over 470 packages in total that are registered, and on Julia 0.3 we have over 300 packages with tests that pass - and we run the tests in all registered packages every night.
Some of my favorite packages (that I didn't make, of course :D) would include
https://github.com/JuliaStats/Distributions.jl
https://github.com/JuliaStats/StatsBase.jl
https://github.com/pluskid/Mocha.jl (deep learning)
https://github.com/stevengj/PyCall.jl
The JuliaOpt stack of optimization packages (http://juliaopt.org)
and then you get fun new ones like https://github.com/anthonyclays/RomanNumerals.jl
Julia's sweet spot seems to be anyone who needs to do statistical work who also has an existing C ABI based environment.
The ability to seamlessly plug in Julia to your production system via C code is a boon for those of us who have to use R in an "offline" capacity and then feed data to production systems.
The downside of alot of the Julia posts is that its not R. R, for better and worse, has many years of bug fixing and library design behind it.
I have yet to stump cran (http://cran.r-project.org/) when looking for a library and after 5+ years of using R I still am amazed that I can learn some new ability that ggplot can do.
My advice to people who ask what language to use is always, I think Julia is still a bit too early on to use unless you already know what code you want to write and are using a C based enviroment.
The reason is that most programmers just want to get one task done. They aren't an expert on the mathematical domain they are trying to use, they just want to plug their values into a library and get their results and maybe print them in a nice format. With Julia I still run into issues too often where you have to second guess the langugage or library and if you aren't fully sure that your math is right you tend to assume you are wrong:)
Otherwise use R. It just works and it will make googling for answers much easier.
EDIT replaced jvm with C ABI. Not sure how I made that big of a typo:(.
For the JVM, Clojure and Incanter seem like a better statistical bet. I heard they're moving toward a 2.0 release of Incanter too...
R is actually a nice language (also with multiple dispatch!), but it's basically useless for any actual numerical work. Too slow. There have been efforts in the past to introduce a JIT compiler, but nothing has become mainline yet. The GC is also very poor.
The actual base system in R is also very small. The built-in vector types helped the language mitigate the issue due to the fact that it's very easy to build a C extension that does the actual computation by just taking the underlying memory pointer (it's actually easier to do than python and it maps much better). But it stops there. If it wasn't for the large swath of statistical packages, R wouldn't have much traction. Not to mention that the code quality of the typical CRAN package is quite low, coming from people with a background in statistics and very little CS knowledge.
The hope with Julia is being able to actually develop methods in Julia itself. I personally looking forward to replace python&c++ with Julia&Rust in the future.
It's been included in the default install of R since version 2.14. You can compile any function you want, and in some cases you can get a speedup of several times. Sure, you're not getting the speed of C, but it's there, it works, and in the future it will lead to much larger speedups.
Some info and timings are here: http://www.rinfinance.com/agenda/2014/talk/LukeTierney.pdf
The experimental version of the compiler can achieve speedups in the neighborhood of 20x.
[1] See page 7 of the presentation I linked above.
Not really. That's an example of one R's rough edges. I did the same thing.
I agree, but Community & Ecosystem are critical to any language's success. R's community is great and getting better [1].
You could make very similar criticisms about JavaScript, which is a lousy language that has accomplished amazing things thanks to third-party extensions and a very active development community.
[1] http://www.tiobe.com/index.php/content/paperinfo/tpci/index....
Rust is filling the same niche I'm using c/c++ for, where I need absolute control of every detail: not necessarily for performance.
I do think the OP doesn't get exceptions, though. Catching and doing something with an exception should be very rare, and almost never happen in library code, unless you've been infected by Java-itis and feel compelled to wrap implentation exceptions at the API level.
Failures are a function of implementation. The stack trace includes the fact that the API was involved. Further wrapping is usually busywork. If action needs to be taken on failure, the code that must not fail should have an exception handler, which can deal with the failure if possible (necessarily failure modes it can enumerate), and retry or abort as required.
Wrapping is particularly pointless with more functional code. When code is idiomatically passed around, library exceptions are frequently from user code passed in, rather than something the library author could exhaustively list.
I fully agree on the need for more testing. We've been trying to make sure that new functionality and all issues reported come with tests where possible and I actually think we've been making progress on that front (and especially compared to two years ago), but we need to do a lot more of this. For a significant number of the core contributors (myself included), Julia is probably the first big project we're working on and we're still learning proper release engineering and making software for a large number of users.
That said, I think we've been pretty good about keeping master functional recently and I don't remember the last time I pulled from master and had something break (I think the travis status analysis is misleading here, because it probably includes development branches as well as instances where the CI system itself is failing - either because travis is down, or because the servers for dependencies are down etc). I also don't think that we've ever left master broken for days at a time when there was an easy fix available (of course bugs that come up and you don't know how to fix can always happen).
On that topic though, I think we have room for improvement. I've been thinking about moving to a pre-commit CI system (you push to a branch and the CI system merges once everything's green), especially now that we have CI coverage for all 3 supported platforms. I'll bring that up on the mailing list.
So much for the Julia core. Packages are a different issue and I agree that we have a problem with package quality. That are a few very high quality packages (I'm thinking e.g. Color and Distributions) and a long tail of packages of varying quality. Iain has been doing great work on this front with http://pkg.julialang.org/pulse.html and making sure that packages keep up with changes in julia and at least pass their own test suite, but there's obviously a lot of work to be done there.
The good thing is though that since these packages are completely decoupled from Julia core, this can be easily parallelized. One of the advantages of Julia is that the core is pretty small (admittedly bigger now than it should be and we need to split some stuff out) and it's in general pretty easy to look at the code which hopefully makes it easy hack on packages. Making high quality packages requires a lot of time and work and since Julia is such a young language, the ecosystem is still immature.
If you have thoughts on what we can do to improve package quality or make things easier on package developers, please let us know, we'd love to hear them.
Finally, I also would like to ask to please, please file bug reports for things that don't work the way you expect them to or for any bugs you encounter. One of the things mentioned in the OP was the REPL rewrite (which I worked on). The original REPL was a messy readline based hack, which really needed to be replaced. Admittedly, readline has a lot of features and I originally only implemented the ones that were part of my workflow, but I fully expected people to file issues for any features they were missing (and they did) and I and others quickly implemented lots of features (including quite a number not present in the original rewrite). Of the REPL issues currently open, I can see only one that may have worked with the original REPL (emacs keybindings). In any case, if you're missing features in the REPL please file an issue.
I'll end with my general disclaimer that while Julia has a lot to offer, it is not yet a polished system and I don't yet recommend using it if that's the experience you want.
Sorry for making this so long
TL;DR I agree, Base needs more testing, but I think we've been making progress on that. Another big issue is package quality, which I'm not sure how to fix. If you encounter bugs please file issues.
It looks like Bindeps is the intended way of building packages that have external dependencies, but last time I checked the "documentation" for bindeps was another Julia package called Cairo, which apparently did a good job.
The other problem is that when Googling around to find the standard way of putting together packages, almost all the results that come back are list posts or things from the issue tracker which tell you the old way of doing things (which no longer works).
Things like how to build things differently for Windows, Linux, OSX, Unix, how to deal with paths, how to use the OpenSUSE binary production system, etc., all need documentation, with examples.
Also needed is documentation on linking against external libraries that have their own dependencies. Where should libraries be canonically installed. How does the linker resolve dependencies (this is especially difficult to work out on Windows).
Including all the .h files for Julia's own shared library dependencies in the repo would also be useful, along with documentation of which versions of external libraries Julia currently uses.
I also find the documentation on how to peg to a commit or tag difficult to understand, as is the documentation on the path to becoming an official Julia package, with explicit examples of what needs to be done at each step.
Finally, I find it annoying that you can do Pkg.update("Mypkg") and Pkg.build("Mypkg") but not Pkg.test("Mypkg") (or is it the other way around, I always forget). One should be able to selectively update, build and test any particular package, without having to do so for every package you currently have installed.
Some of my comments might be out-of-date by now. I'm simply trying to recall some of the issues I had with building a package when I last put effort into that part of my development cycle.
Pkg.update() is currently designed to update the local METADATA snapshot and then update all packages. You can also update one package at a time manually with Pkg.checkout.
It also helps to have a culture of quickly rolling back breaking changes when they do slip through.
[1] https://github.com/JuliaLang/julia#required-build-tools-and-...
Generally the packages that aim for higher performance are more difficult to install, which makes sense.
Combinations: http://www.omegahat.org/Combinations/
Cranvas: https://github.com/ggobi/cranvas
edit: and if, god forbid, cran kicks off one of the packages you're using or an upstream package for a package you're using, then you've got to track it down and install it on your own. (tikzDevice is the one I'm thinking of.)
Not to armchair quarterback, I know it is a hard thing to get there and you guys are all busy. Just my 2 cents.
(And I think Julia is bloody brilliant, fwiw)
Having significant indentation like Python is another option, but one I've never cared much for – it makes cutting and pasting code a bit of a nightmare, among other things.
Sadly, that makes it hard to ask the question "what methods can I call on an object of this type?" because the answer might include functions with that type in any parameter position. In many cases that wouldn't be a meaningful answer - as in the case of, say, a numeric type which could be used for all kinds of indexing, iteration, and offset methods involving collections, IO handles, etc.
You might benefit a bit more from using the 'apropos' builtin to lookup functions based on simple keyword matching. Given a type name it lists methods that take or emit objects of that type (which may or may not be exactly what you want, but it's a starting point at least).