Hacker News Reader

Comment by KenoFischer | Hacker News Reader

Happy to address these points:

  - Startup performance/memory usage

Yes, we are definitely very acutely aware of these. Julia is not currently optimized for frequently run short scripts. That's the price on pays for having to bring up the entire runtime system (initializing the compiler, RNG, external libraries etc). The good news is that there will be a solution to this soon, which is to statically compile your julia program. The area where this really comes up for most people using is package load times. We're very actively working on making that faster.

  - Syntax

A little subjective, so not sure how much I can say here. I can say that I'm not a huge fan of our multi-line comment syntax. It's not entirely clear what a better syntax would be though (the original issue on this had some suggestions, but some of them were worse).

  - One-based indexing

I think there has been plenty said on this topic, though interestingly this is one of the only times I've seen the argument made in a way that I actually agree with. That said, I do think there is an easy way to deal with this though. For packages that needs arrays of indices, it would be quite easy to define an `IndexArray` type that does the translation automatically.

  - String Formatting

Yep, you're right, it's a mess. It'll have to be cleaned up.

  - Unsafe C Interface

There's two projects (Clang.jl and Cxx.jl) which can help with this. The former automatically generates ccall definitions for you, the latter just parses the header and generates the call directly.

  - Slowing down in development

I'm really not sure where that impression comes from. Perhaps it is that we're adding fewer features, but rather working on cleaning up existing features. Also, I personally at least have been doing a lot of work outside of base (particularly on the debugger). Not sure. Would love to know.

View on HN

Thanks for the detailed response. I hope that my post wasn't too harsh, the intent was mostly to attract attention to the current issues not to undermine the great work that you and others have been doing. I'm glad that many of the issues that I mentioned are being addressed. Maybe I'll give Julia another go in some time =).

The question of syntax is subjective of course. From the set {C-like, Python, MATLAB} I'd definitely like to see more influence of the first two and less from the last one.

One-based indexing is also used in Fortran, which seems to be used in a great deal of numerical computing even today. Additionally, BLAS/LAPACK is an important linear algebra library written in Fortan.

I am somewhat confused by your discussion of startup times. Since Julia is a "programming language for technical computing", what scenario are you imagining where startup times would be a significant concern?

No less than Dykstra has weighed in on the numbering of arrays.

http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF

Having worked with both, I'm inclined to prefer 0 based addressing in most cases. It's slightly less intuitive, but it generally leads to cleaner code.

No less than Dykstra spouted a half hearted list of some properties and then made a proclamation based entirely on aesthetics.

no less than dykstra was convicted of grand theft auto. https://en.wikipedia.org/wiki/Lenny_Dykstra

I think you mean Dijkstra, who did say that, because of an argument with mathematicians about indexing from 1.

In Dutch, ij and ÿ are interchangeable.

My Dutch professor in college actually wrote out the "ij", but the tail of the "i" connected with the "j", making it appear to be "ÿ". It kind of blew my mind the first time I saw him write it out haha

In cursive, it is often indistinguishable too.

In print, NO. You really can't interchange the two. As already said, in handwriting they may look similar, however we would never mean the actual ÿ, since it simply is something else which is used in Greek and French (and others).

There's even a section about this on wikipedia: https://en.m.wikipedia.org/wiki/IJ_(digraph) subsection "technical details".

At the very least it happens in Flanders. I've seen ijs ("ice") capitalized in cursive as Ys[0] on an ice truck, for example.

Edit: from Wikipedia[1]: "It used to be common, in particular when writing in capitals, to write Y instead of IJ."

So it's an obsolete practice...

----

0. something along those lines: http://alphabetprintables.org/alphabet_printables_cursive/up...

1. https://en.wikipedia.org/wiki/IJ_%28digraph%29

That argument never felt very convincing to me. Perhaps because it really is about aesthetics, not some deep philosophical ideas connected to numbers.

I see it as rather simple. If you're listing off-sets, starting at 0 makes sense, because there'll be something there. But if you're counting elements starting at 1 makes more sense. I don't have 9 fingers offset from my first one, which I call finger zero; I have ten fingers, starting with the first one.

A set with no members is the zero set, once you add one element, you have a set of one. An empty list contains no elements. It does not have a first member, because it's empty. Add an element, and it will have one member, and it will be in the first position. Not in the zeroeth offset from the beginning.

You can the numeric names of letters of a string in an array, but I don't see that as a compelling reason for naming the first letter of a word, the letter which is at the zero-eth offset from where the word is held in memory. At best, it's a low level optimization, at worst it's just a leaky abstraction. Maybe you want to store the length of the word there (like in Pascal), for a different trade-off in terms of what is optimized.

Indexing is not for counting elements, it's to reach a particular one.

I still find it more useful to name it my tenth finger, rather than the ninth from the first. And I'm fine with my middle finger, finger number three - being the one that sits between the pairs of fingers made up of finger number one and two, and fingers four and five. I don't really see how it's any better to count up from the zeroeth offset of the first finger to the second offset to get to the middle finger.

I don't buy the Dykstra argument,1 base is more intuitive for me and for most non programmers, I would guess. Funny though how Americans use 1 based indexing for building floor numbers, but Europeans use zero based i.e. ground floor is zero in a European lift, 1 in a US lift.

Not just FORTRAN, but R and MATLAB also use one-based indexing. It's also, at least historically, the convention for matrix notation. IMO do whatever attracts more users, as that is what Julia needs most. Without a large community moving from MATLAB, R, and other languages, Julia will never take off.

Erlang also.

In most functional languages using list indices are an anti-pattern. Pattern matching and generalized iteration is a much more elegant way to handle most things you would use an index for.

Well, sort of. Erlang has the array module, which does actually index at 0, intentionally to feel like an array from another language.

The primitive collection types index at 1, but, as you said, are almost never indexed that way. I'm not sure the motivation as to why, but the fact it feels clunky to use them that way is a benefit, as it raises resistance when you're using them wrong (as indexing them almost always is).

BLAS is an API, more or less.

https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra...

And the Fortran-convention BLAS API is quite a bit more widely used than the CBLAS API. idamax in the Fortran API, and pivot vectors in LAPACK factorizations, return 1-based indices.

> what scenario are you imagining where startup times would be a significant concern?

Starting a REPL or running a computation that doesn't take long time, adjusting parameters, re-running. Startup time may not be too big in absolute numbers but it's noticeable and adds up quickly especially if you start using more packages rather than toy programs that I used as an example.

Sorry if I was being vague, but what situation would you re-run a short computation by executing the whole program again instead of using subroutines? Are we talking about a data scientist performing initial, exploratory analysis on a very small subset of data?

I agree. I'd estimate I restart the REPL about three or four times a day while developing. My analysis runs are minutes or longer, so a few seconds spinning up Julia+packages just doesn't ammount to much.

> Are we talking about a data scientist performing initial, exploratory analysis on a very small subset of data?

Yes, that's one example. Also when debugging one usually uses small data sets. There are plenty of cases where runtime is short.

I think the problem is that Julia is somewhat vague on how it should be used. If it stated explicitly that it is intended to be used in MATLAB-like fashion with one long-running instance that would save people from trying to use it as Python or other dynamic language.

In that case, it should seem like a fix is as easy as keeping an IJulia kernel running in the background all day.

Ok that puts some bounds on it; we are talking 1/10ths of seconds as significant? Or 1/2 seconds? There is work on this (https://github.com/JuliaLang/IJulia.jl/issues/346) and later versions of Julia 0.4 feel pretty snappy to me compared to R (which is the other tool I generally use nowadays)

Fortran was meant to be a high-level language at the time, so chose 1-based indexing. When C and lower-level languages came on the scene they reverted it back to zero-based, since this is more appropriate at the lower hardware level. CBLAS is zero-based.

Intel's Math Kernel Library, which is a performant math library hand-tuned for Intel processors is zero-based in MKL-CBLAS.

It all depends on what you are familiar with, and staying consistent in use. I just use zero-based indexing in J and C and my numerical low-level work.

It is all a matter of what the compiler does, array accesses should generate the same Assembly instructions.

I have used quite a few languages whose base index could be 0, 1 or whatever I choosed. Even enumerations.

Fortran has the rather unusual feature that you can change the indexing to an arbitrary offset. You can declare arrays to be indexed from 0 if you want, but it defaults to 1 and almost nobody uses this feature for obvious reasons.

BASIC did as well IIRC.

Pascal as well I think. I dunno which was the first to support it, but it's not exactly a popular thing in modern programming.

It was already there in Algol-60.

Pascal even let you index with non-numeric types, so you could have an array indexed by the characters a to z, for example.

I index from -N:N so that 0 is in the centre of my computational boxes.

Yeah, something like this is the intended use, but it isn't used much at least in the Fortran I've seen. Mostly because it's really confusing for someone reading your code, as once the array is declared there's no indication what the index range is.

If distances used 1 based indexing:

    metric distance conversion chart:

    cm     m       km
     1     1.00     1.00000
     2     1.01     1.00001
     3     1.02     1.00002
    ...
    101    2.00     1.00100
    ...
 100000 1000.99     1.99999

The ratios between the values aren't fixed now; we can't go from cm to m just by scaling by 100. We must subtract, scale then add.

One based indexing falls apart if you have to index a region of storage as bits, bytes and words at the same time.

Distance doesn't use indexing at all, because measures are not indexes, so it doesn't make sense to say "if distances used 1-based indexing".

OTOH, for something physical that actually is indexing, or at least closely analogous to it, we could ask "what if principal quantum numbers used 1-based indexing". But, then, the answer would be "things would look exactly like they do now, because it already does."

Distances can be quantized. If you're designing something in your CAD tool, you can set a grid to, say 1cm, and never place anything on a fraction of a cm. The coordinates of points in the design are then de facto addresses.

Indices can be regarded as measures. We speak about an array having a "size" or "length": that is measurement language. Something is "3 words wide": ditto.

A given record in a file can be 25 words from the beginning, or 100 bytes, or 800 bits. All of these tell us how much storage immediately precedes that record and we can easily convert among them.

If indices support calculation, they should be displacements, and displacements should originate at zero.

Indices not intended for calculation (beyond simple successor/predecessor, perhaps) can place items into correspondence with any ordered set: natural numbers, letters of the alphabet, and so on. This is where we can get away with 1 based.

Indices not intented for any calculation whatsoever can use a set: like associating character strings with objects via an "associative" array or whatever.

But that's the point, once you think of indexes as distance things make much more sense.

The index is the distance from the first element. Thus, the third element is two elements away from the first, thus it has index 2.

I think the only reason zero-based indexing is not the standard everywhere is because some people (usually non-programmers) have a problem with the idea that the fifth element has index four.

I am suggesting that this is a linguistic problem that has messed up programming.

Consequently, when those people do music, they have to accept that something called an octave has seven notes, and inverting an interval is based on the "rule of nine", thanks to the way they defined intervals as one based.

Except that's counting, not indexing. It doesn't make sense to compare the two. It's more akin to the "is the first floor the ground floor or the one above it"-debate.

Perhaps you mean it's displacing not indexing?

Counting is indexing!

What is it that you do when you count items in a set? You put them into correspondence with the natural numbers, indicating each one as 1, 2, 3, ... The last integer is the count.

Yes, it is related to the "is the first floor ground, or the one above it"; and it's clear that some programming languages take their cue from stairs and elevators.

Nobody ever has to calculate "what is the floor twice as high as this one?" Moreover, people are unfazed by 13 missing.

(I haven't seen a language that omits 13 from indexing, fortunately.)

Since we can probably agree that you're not indexing distances or other units of measurement (you can make a piece of string of length sprt(2) centimeters or other irrational lengths!), let's skip the terminology argument. That wasn't my point.

Is skipping the 13th floor a US thing? I don't recall ever seeing it in the UK or Germany.

Yes, in the US, you will almost never see a 13th floor.

It's ironic, though, that most languages report 1-based source code line numbers in their error diagnostics, regardless of how they handle indexing. So that does say something.

And their documentation has section numbers like 1.1, 1.2, ... 5.3.3.

The current Julia version is 0.4.5 =).

I liked how the Template Numerical Toolkit implemented one-based indexing:

    // Construct matrix. Elements are not initialized.
    // To initialize all elements to zero use
    // Matrix<double> a(2, 2, 0.0).
    Matrix<double> a(2, 2);
    
    // Assign elements to first row using
    // Fortran-style one-based indexing.
    a(1,1) = 0.5; a(1,2) = 1.0;
    
    // Assign elements to second row using
    // C-style zero-based indexing.
    a[1][0] = 1.5; a[1][1] = 2.0;

-- http://www.b-a-h.com/software/cpp/scppnt.html

This syntax is available in julia as well, but I'm not sure it's a great idea to encourage mixing the two indexing behaviors, even if they have different syntax. As I hinted in the original reply, I have seen very few cases where the choice of index offset actually makes a difference. For example, loops over indices generally use `eachindex` which doesn't care about your choice of index base.

That's interesting - thanks. I don't think anyone is eager to mix indexing behaviors either.

I just realized that when I said, "this syntax is available", people might have understood it to mean, "you can use this syntax to index arrays" (which is not true for the array type defined in base). What I meant was, "you can define an array type with this indexing behavior without changing the language (since you can override the behavior of both () and [] on a particular type)".

Actually, for the sort of low-level bit twiddling code that I frequently do, 1-based indexing complicates the code greatly. I also need to pass structures that contain indices back and forth to C, so the 1-based indices cause extra overhead to constantly add or subtract the size of the element. I find that it's the largest cause of bugs in my code (which is why I really want Gallium working!)

Keno - take a look at what I did in https://github.com/ScottPJones/StringUtils.jl (building on the backs of giants, such as Tom Breloff, Dahua Lin, and others, incorporating some code from Formatting.jl). It solves problems I've had both with string literals, formatting, and outputting funny Unicode characters (without having to use UTF-8 in my source code).

I really like your Cxx.jl, that does seem to answer the issue raised here (are there any good features in pybind11 though that aren't already in Cxx.jl? that might be a good source for ideas on how to improve Cxx.jl if not)

> For packages that needs arrays of indices, it would be quite easy to define an `IndexArray` type that does the translation automatically.

(just to add: there is currently an open pull request from a core array-focused developer adding such support to base)

I believe this is the PR in question: https://github.com/JuliaLang/julia/pull/16260

Regarding the slowdown in development, perhaps it's a seasonal variation connected to GSoC. For example, notice a drop in postings since October in https://groups.google.com/forum/#!aboutgroup/julia-dev . This seems to be more or less consistent with the observation in http://www.davideaversa.it/2015/12/the-most-promising-langua..., but that's just a guess.

No summer of code students have had projects that involved work primarily on the core language since 2013. Not that many posts happen on julia-dev because most of the development communication happens on github.

For better or worse, most development discussion happens on GitHub rather than julia-dev. I've been catching up on GitHub threads recently, and the progress towards 0.5 is really impressive -- both in the language itself, and in the tooling (especially the debugger). I think the nature of priorities has changed somewhat as the language matures, but I disagree that development has slowed. (disclosure: I'm a Julia committer, but have been a bit out of the loop for some months now)

There is certainly a need for better interim communication on progress between releases, to let people know what is happening without reading every GitHub thread.

I do think it's seasonal, however not connected to GSoC. Many of the major contributors are (or were) students (Julia seems to be detrimental to people finishing their PhDs, except in the case of Jeff Bezanson, where in some sense he crowdsourced a lot of the research, as Julia was the topic of his thesis ;-) ) Many who aren't students are professors. Definite drop off when people are taking classes, exams, teaching, etc.

Might be as simple as creating a static and dynamic profile for Julia. I suggested this to developers of one language project before. Maybe Julia. A single keyword or declaration near top of the file tells it to compile for fast loading with no need for functions that change things at runtime. Or compiler option.

We do have various compiler flags, but the problem on startup is the initialization of external libraries and deserialization of the system image, primarily.

The thing about 1 based indexing is that it's a kind of in your face "this is different" decision from the point of view of a programmers of most popular languages. To be honest I wouldn't want to start investing my time into a language where people who proposed 1 based indexing are making design decisions. It's not that I think they are incompetent but it's clear they care way more about some different world than about my programming world and are ready to make my life miserable stating the point.

Now, I don't know if people from that different world (Fortran, Matlab, some other languages used in academia maybe) would feel the same way about 0 based indexing but it certainly sends the message to programmers outside those domains.

R and Mathematica are also 1 based. And R really is popular (http://www.tiobe.com/tiobe_index?page=index - ok, popularity is droping right now, likely b/c of Julia ;-))

Julia seems like it's meant to be friendly to people who know MATLAB, Python, and R (even Fortran). R and Python are really comfortable with working with data science and that's what Julia is geared toward. I have no experience with MATLAB, though. Julia's programming "world" may indeed be a different programming world than C programmers.

I'm not defending Julia per se. I've been patiently observing from the sidelines to see how the language shapes up.

R is ranked lower year-on-year, but if you click into the actual rating time series you can see that R is on a tear into new highs of popularity. For that matter so is MATLAB. Julia is still buried in the next 50 list.

BTW i'm a julia proponent, I'm just starting into it but I like a lot about it.

So is Lua.

Lua tables are more interesting than that. You can have an table that is indexed by any supported type. That means tables indexed by positive or negative integers, floating point values, references to other objects or other tables, true, false, strings, coroutines, and external memory references can all be indexes. You can also have a single table indexed by any combination of these.

Isn't that technically a hashtable and not an array?

In Lua the same data structure is used for both. The implementation optimizes the data structure like an array when you use a range of non-negative integers as indexes.

>"popularity is droping right now, likely b/c of Julia"

I don't see how you concluded that? http://www.tiobe.com/tiobe_index?page=R

You shouldn't be so biased. Having one based indexing makes translating numerical recipes from pure math textbooks (where vectors and matrices are generally one-based) simpler and less error prone. I say this as someone who chafes at Julia's one based indexing as a matter of professional practice.

It's very error prone if you implemented algorithms using 0 based indexing your whole programming life. You are just bound to make mistakes if suddenly it's 1 based indexing.

I am not saying 0 based indexing is technically superior (I really have no opinion on that). I could compare it to switching positions of brake and accelerator in a car - you can talk about brake on the right design being superior all you want but the fatal crashes will happen if you implement it.

Of course, zero-based indexing is prone to error if you've grown up using natural language ordinals your whole life, and get business requirements based in them both of which are stunningly common of real developers and real development tasks.

This is exactly why all of my off by one errors happen in zero-based indexing languages. I need 1-based languages for data analysis, and 0-based languages for working on instrumentation.

0-based indexing is technically superior, because it makes offsets for memory addressing simpler.

Well unless your array is more than just x-many addresses in a row, like if the array carries metadata related to length (like every language but C) or if it's a sparse array (hint sparse arrays are super common in numerical work).

Does 1 based indexing actually matter? It does if you are doing index arithmetic for rolling your own multidimensional arrays for example, but one shouldn't do those kind of things anyway.

One thing to keep in mind is that scientific computing languages often have people using them who are not programmers, and who will not be "rolling their own multidimensional arrays."

If they're used to R, MATLAB, or basically any statistics programming/scripting language ever, not only will they be expecting 1-based indexing, they might not even know that that's a thing they have to think about.

>> they be expecting 1-based indexing, they might not even know that that's a thing they have to think about

The truth of the matter is that this is not a point that anyone should ever mention as either a pro or con of any language. It's simply a fact that you're going to see one or the other, and it's useless to complain about which one is present in any given language. It's a one-time discovery as to which you're working with. People try to make it out to be a debilitating factor, when in reality it's a non-issue.

There are tons of things which you need an index for when iterating over an array. Do something different for every 4th element but then something else if it's 16th. Keep doing something for triplets of elements unless you run into specific index which is divisible by 7, then start taking them one by one. Iterate but go 5 elements back if you encounter something in some state. There are a lot of algorithms which take advantage of array being an indexed data structure with an order and not just plain container without structure.

The problem doesn't just go away with for each style loops. Python added enumerate for this reason but it really is quite clumsy for anything more complicated (and you have to choose if it's indexing from 0 or 1 anyway).

To be honest I wouldn't want to start investing my time into a language where people who proposed 1 based indexing are making design decisions.

Possibly because you're used to working with languages like C, C++, Java, Pascal, Javascript, Python, etc. But in the world of languages tied closely to scientific / mathematical programming (Matlab/Octave, R, etc.) 1-based indexing is the norm. If you'd "grown up" so to speak, in that world, you'd probably find 0-based indexing distasteful.

Pascal (and Ada) actually expects you to specify the both the first and last index foo[1..10] or foo[0..9] are both valid arrays (though I believe 1 is the traditional first index) which is handy when you do run into situations where one or the other is a poor fit for an algorithm.

Yes, my argument is purely about habits. 1-based indexing is a potent trap for programmers who are used to implementing stuff in 0-based indexing world.

Around 3 decades ago when started with BASIC, Pascal/Delphi 1-based index was pretty much the norm and off-by-1 mistakes were significantly more dominant.

Switching to zero based index Java/Javascript/C felt a lot more natural (no need for upper bound, always include->exclusive when it comes to bounds). Many algorithms benefit greatly from as well (hashtables + bitwise AND, for instance)

Virtually I do not recall making off-by-one errors while working with Java. Whilst it could be attributed to personal experience, the loops construct "for (int i=0; i<length; i++)" or "for(int i=length; i-->0;)" appear quite intuitive. Strict bounds (i.e. < or >) feel more elegant as well.

Seeing, using both type of indexing, 0-based one would a winner by large margin in my book.

I don't really get the difference. I'm in academia i guess (grad school) and use MATLAB a bunch, as well as Python (not a CS major). I don't think it's that much work to switch between the two, and I don't understand why there's such a division. Why is this such a huge deal?

It's not. It's nothing more than bike-shedding language design.

It's an easy thing to discuss, and everyone is an expert. Meanwhile, the real problems get ignored.

> - Slowing down in development

I think part of this problem is the community, multiple people have reported having bad interactions with core language devs. Also the policies for inclusion of features, how to propose features, how decisions are made with respect to the code base, etc. all seem poorly documented.

> I think part of this problem is the community, multiple people have reported having bad interactions with core language devs.

I can only think of a couple of instance where I've heard anything like this.

The first and most prominent is Dan Luu. I'm very sad about losing him from the community. I've met him a couple of times in person, I still read his blog and I very much respect him. I don't think any of us have the full story of what happened there, but I sincerely hope that time may be able to smooth things over.

The second was in an HN comment that I just went back to find, and discovered that is was you. I'm sorry if you had a bad experience with the community. I know my perspective is biased, but I've rarely interacted with a community that's as passionate, helpful and friendly as the julia community (LLVM is up there as well).

As for documenting the process, I agree there could be some improvement there. We do have a contributors guide (CONTRIBUTING.md in the main repository). As for policies for feature inclusion, we've considered having more formal code owners for parts of the code base (right now there's mostly de facto code owners for various parts), but haven't found it necessary so far. I definitely expect some of this to be discussed at JuliaCon in June.

I remember plenty of cases I got simply from observing the Julia repo for a few months, I could list them if you like... But I've been told by Julia contributors before that discussing this on HN is not an appropriate place. So we can discuss it, where I start linking to public examples on HN, or you can ignore my opinion on a problematic community and we can not discuss it.

As to the code standards, my problem isn't with the CONTRIBUTING.md; that seems fine. It's how people make long term suggestions about fixing lasting problems. As an example people who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned. Important improvements to the language go in circles (often on JULEP tagged issues) especially any large improvement that would require multiple people and lots of dev work and seem to gain little traction unless a core dev just takes the time to just do it; rather than a timeline to reach consensus and then design a plan for implementing it.

As smaller examples. There was a push to increase test coverage and then months later commits plummeted the code coverage numbers and no one seemed to care. Sections of important code are uncommented, undocumented, and (mostly) untested on purpose. There are modularity problems and software architecture limitations inherent in the language design that go unaddressed. Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, "interfaces" that aren't really. And these are just the ones without serious work behind them I can remember off the top of my head (the debugger and threading I acknowledge being worked on seriously). Though things being worked on seriously suffer problems as I mentioned earlier.

Issues like these cause many of the original poster's problems. Julia is a great language, but the community seems unable to take the steps it needs to to actually make a "Tier 1" general purpose language (e.g. like python, go, clojure etc.). Setup a serious set of policies regarding language proposals; setup a transparent community policy and actually follow it; acknowledge that certain development idiosyncrasies aren't going to fly anymore; take software architecture concerns seriously; remove the toxic core language dev (that to my count has cost you at least 4 serious contributors if not more).

Julia is likely "slowing down" relative to the expected development (which many people have observed) likely (by my thinking) because it is failing to attract/retain the developers it should be as a promising language (in my opinion) because of it's poor development and community standards.

But hey, these are just my suggestions, I don't have any skin in this game. I just wish Julia was as good as it promised to be, and I'm a bit bitter it isn't for human, rather than technological or financial, reasons.

Edit: Sorry I expanded, it felt unfair to not be more complete about it.

> I remember plenty of cases I got simply from observing the Julia repo for a few months, I could list them if you like... But I've been told by Julia contributors before that discussing this on HN is not an appropriate place. So we can discuss it, where I start linking to public examples on HN, or you can ignore my opinion and we can not discuss it.

I do think HN is not the right forum to link to individual comments and call out people for their behavior. Perfectly happy to discuss the technical issues though. If you would would still like to discuss, or at least bring such instances to my attention, please do feel free to send me an email.

> People who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned.

There are several recent examples where people have suggested such things, e.g. They were neither ignored nor shouted down nor banned. https://github.com/JuliaLang/julia/issues/16357 https://github.com/JuliaLang/julia/pull/16070

> Important improvements to the language go in circles on JULEP tagged issues namely any large improvement that would require multiple people and lots of dev work seems to gain little traction unless a core dev just takes the time to do it; rather than a timeline to reach consensus and then design a plan for implementing it.

I don't understand what exactly the criticism is. At first I thought it was about, too much dicussion, but then it seemed to be about too little dev time, please do clarify.

> There was a push to increase test coverage and then a few months later commits plummeted the code coverage numbers and no one seemed to care.

I'm not really sure which instance you're referring to, but last time there was a major drop in coverage, it was investigated, found to be a bug in the instrumentation (which was fixed). I don't really think it's fair to say that people don't care. There's somewhat of a tooling problem here, since we can't run the coverage tests on travis, so we don't get them integrated in the GitHub UI, but people do look at them and add tests as appropriate.

> Sections of important code are uncommented, undocumented, and untested on purpose.

Yes, there are undocumented and uncommented sections in the code base, but I wouldn't say that they are so on purpose. I do admit to having added such hacks in the past (and not documented them because they were gonna go away soon after), because I needed them in outside packages, but people don't let me do that anymore ;).

> Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, interfaces without any sort of type enhancement.

These are fair points, which I'm sure you've seen the issues about. The problem is the availability of developer time, not some sort of unwillingness to fix problems. Putting together the road map of what goes into each release and prioritizing are very hard, because there's just so much that could be worked on.

> setup a transparent community policy and actually follow it;

This is being discussed (as part of larger discussions around community governance) and as I mentioned will likely be a topic at JuliaCon.

I haven't disappeared (even though somebody just might have preferred that I had done so), I still love the Julia language (even though I think there are some warts, some flab that can be lost, etc.), am still programming in Julia pretty uch 24/7, have been very active in trying to help people new to Julia (on Gitter, StackOverflow, Quora, julia-users), even teaching it to my kids, promoting it on Twitter, trying to help improve the language as much as I can without being able to submit issues or PRs on GitHub, and contributing what I can of my own work under MIT license on GitHub. I'm also attending JuliaCon 2016 this summer, will just have to see what my reception there will be.

> There are several recent examples where people have suggested such things

That's great but amount to a couple of functions, the larger issues addressing more serious refactorings never get off the ground (Base contains multiple thousands of symbols (not even counting how in a multiple dispatch language one symbol can contain hundreds of definitions) common lisp, by all accounts a sprawling language with multiple dispatch, has 978).

> I don't understand what exactly the criticism is. At first I thought it was about, too much dicussion, but then it seemed to be about too little dev time, please do clarify.

Both. Neither. It's about how there is unending discussion until a core dev just does it. This is not an efficient use of anyone's time. Set the scope of an issue, discuss, decide on a course of action, layout issues. This allows a core dev to get their input in place, but then not have to actually wait for them to have the time to do the whole thing themselves.

> There's somewhat of a tooling problem here, since we can't run the coverage tests on travis, so we don't get them integrated in the GitHub UI, but people do look at them and add tests as appropriate.

The number seems to fluctuate randomly from 83% in January, to 11%, 45%, 65%, 75%, 11%, 81% now. Which is a net loss and wholly sporadic. How does anyone know what to write tests for when coverage bounces all over the place? Also what good are the tests if no one takes the time to use them anyway? You have tests but they seem pretty useless with how you are using them. I understand it's a tooling issue, but a metric isn't very good if no one is bothering to use it, why waste the time to not use it.

> but I wouldn't say that they are so on purpose

"Look at the code and you'll understand." is the jist of the documentation of a 10000+ SLOC base used at the heart of Julia. Sure I could read all that, but I'd rather have a bit of documentation. Also the tests for this amount to it's examples.

> not some sort of unwillingness to fix problems

I disagree on at least 1 of them. Typed exception handling was flat out rejected because it's too much like Java by key devs; forget the fact everyone does it anyway using reflection. Interfaces and modules don't seem to make any progress because of some people complaining about similar issues.

> Putting together the road map of what goes into each release and prioritizing are very hard, because there's just so much that could be worked on.

Then stop wasting dev time and work on the community policies that make it easier to retain devs.

> I do think HN is not the right forum to link to individual comments and call out people for their behavior.

Fair enough, my problem is that the project continually looses what appear to be serious contributors to a toxic developer (admittedly all second hand to me; but there are public examples and some private reports I have; including the person who first showed me Julia years ago). For all that it is an interesting language with plenty of problems I would enjoy fixing and taking the initiative on I simply don't have a desire to want to contribute. Again mostly due to politics. Which just sort of makes me bitter you fail to acknowledge the problems.

Like why should I take lots of time (I'm fine doing hard things like reading undocumented code bases if it's worth it (large sections of the core compiler code due to it's lack of comments or documentation are hard to read)) to help you fix technical problems, when you have social problems that are noticeable to anyone paying close enough attention to the community.

Also, thanks for taking the time to listen to my rants. I'm just disappointed.

> Which just sort of makes me bitter you fail to acknowledge the problems.

I'm perfectly happy to acknowledge that there have been conflicts in the community. However, I maintain that the overwhelming majority of interactions in the community are immensely positive. Please do also consider that there may be parts of the story that you are not seeing in the public issues. I do hope you will consider giving the community a second chance. You have quite clearly identified some of the major technical challenges we face and we'd love to hear any ideas you have to address them.

Also, I do thank you for taking the time to give feedback.

I have never had a bad interaction with the core language devs. OTOH, they are passionate about the language; I've found all the core developers I've met very helpful.

I'm referring to one specific core dev. However they all know each other from MIT so it makes it difficult for them to deal with that fact. I can point to 3 public examples, in addition to a couple of private reports about problems which allow me to be certain. But admittedly it's all second hand to me (from people who work close to the language).

I've been using julia for a little more than two years now. I've been subscribed to all the mailing lists and regularly read the github issues. Not all of them, but a fair share. I've never, ever seen any behavior by any of the core devs that one could even remotely describe as toxic, rude or anything like that. The community is actually really helpful and supportive.

I have seen this point about a toxic dev been made before in some blog post. Back then I followed the story up because it seemed so at odds with my experience with the core devs. It took a bit of googling and following links, but in the end, in my mind, there was simply nothing to the whole story. The supposedly rude behavior was not at all rude, imho.

I don't really know where these allegations come from, but I find this kind of "I've heard second hand that there is a toxic dev" inappropriate. If someone has a problem with someone, make it explicit, post the email that you dislike, so that others can judge themselves. But these vague accusations are not helpful, and at least from my point of view entirely at odds with how I have perceived the behavior of the core devs over the last two years.

I should say that I'm not part of the MIT crowd. I've never met or talked with any of the core devs and don't know them beyond reading their emails on mailing lists and sporadic interactions on github.

The rude behavior definitely exists, I can point out a number of examples if you send me your e-mail address (you can look me up, @ScottPJones on GitHub, easily enough). I've seen that that sort of disrespectful behavior tends to spread in the community, unfortunately, when nobody dares call out one person for their comments and actions, because of their position.

But then - as a bystander - let me mention that I once wondered about your insistent replies in some gh issue. God, can't he stop, I thought, and, wow, they have patience. Fortunately such a situation is rare and I'm very often impressed by the intelligent friendly conversations I see. A lot to learn.

Yes, a year ago I'd never worked on any open source project, didn't use GitHub, Google Groups, Twitter, StackOverflow, or any of the other programmer social media sites, and was a total newbie about Julia, using git, and dealing with an open source community. Since I had convinced the other people at the startup where I am working that using Julia would be better for us than a mashup of Python, C, C++14, R or Octave, when I started encountering serious bugs and performance problems in areas that were critical to our development, I did get very insistent, esp. where I had lots of previous experience in the area. Instead of simply complaining about the problems, I submitted issues, wrote documentation, made PRs to fix the issues (in the Scheme parser code, in the core C code, in the C Unicode handling module, esp. to the string handling code in Base, and helped improve the coverage by adding a lot of unit tests). It was a major learning process for me, learning best practices for how to break apart a PR into smaller chunks (even though accepting huge PRs from other developers happens all the time), trying to focus on single issues in a PR, and trying to improve my own communications skills on social media. Between the time I started until early September when v0.4 was released, I'd even reached the point of being the #6 contributor to JuliaLang/julia (#26 all time out of some >400 contributors), so I don't think anybody could possibly accuse me of not having putting my time and effort where my mouth was, as far as fixing problems I encountered. Having gotten the critical issues fixed and into v0.4, so we had a reasonably stable base for our product, I was able to concentrate wholly on our product, and so stopped being so pushy about getting things fixed. ;) I still think there are some real systemic problems that need to be addressed, in how things are done, and hopefully some time will be spent to deal with that at this summer's JuliaCon 2016.

You know, I read some of the harsh comments you got from people, and they may have been right to say these things, I don't know. But kudos for taking it in stride and maintaining a positive attitude. Good luck in your projects.

Thanks. I've never had any problems with constructive criticism, no matter how harsh it might seem (often that is where you learn the most). The problem is when people make disrespectful, disparaging, insulting remarks (which is against both the NumFocus & Julia community standards), and nothing is done about it, which has ended up driving away several people already. Hopefully that can be addressed.

Thank you for this reply, I also wish you luck with your project, joy at the JuliaCon and I'm glad that 0.4 worked :-)

Maybe a source of discontent is that startups are using Julia 'for real' and are under pressure to deliver. On the other side Julia is still developing. Core devs work feverishly but some issues just need 'time to brew'. Quiet thinking/coding alone or in small circles.

IIRC on a video of Alan Edelman he ~ said that he was involved in HPC for 30 years and to this day they weren't that successful and don't know how to do it. Julia is trying to do it in a completely different way. - Considering 30 years, does it matter if Julia takes 'her' time to be crowned?

I followed your interactions with the core devs, at least the public ones. I think the core devs handled the situation with an enormous amount of patience and tact.

Perhaps you might consider adding a note to your first post making it clear that you are identifying a problem with one other person rather than the whole community and that this is all second hand rather than your own experience?

You realize edit timers are a thing right?

I can only speak for myself, I attended an evening "getting started with Julia" event at MIT a few years ago, the devs there were very tolerant of me not being able to do some of the exercises and sat with me and helped me understand. Then they gave me Pizza. I've moaned sometimes about some of the decisions that they've made and they have always responded to me and showed me that they have a good reason for doing what they are doing - even if I still think that might be wrong.

Reply on news.ycombinator.com