Hacker News Reader

Comment by 3JPLW | Hacker News Reader

There are many many tomes of discussion that have led to this point (I'm mbauman).

In short, there is an inherent disconnect between arrays-as-containers and matrices-as-linear-algebra-operators. There are many ways that you can define array operations such that the universe is closed and self-consistent, but there always seems to be some awkwardness. Right now, Julia uses some Matlab-ish semantics (sometimes ignoring trailing singleton dimensions), but there is an interaction with Julia's types that makes the status quo a little strange.

So there are "just" two major breaking changes proposed here: changing our array semantics universe to be more APL-like (instead of Matlabish), and making indexing return a view instead of a copy.

View on HN

A little bit of history here. Back when we started Julia 6 years ago, I personally was in a much more of a Matlab/Octave frame of mind from my own experience. That is why the early array implementation followed that style.

We had no idea where Julia would go. In some sense, it has been a personal journey of maturity in my own thinking, and if I can take the liberty to say so, the same is true for the community as well.

Has your stance on one-based array indexing changed in those 6 years? How many off-by-one issues have you encountered?

I'm not from a Matlab/Octave background, so one-based indexing feels really foreign to me.

For what it's worth, I've ran into this a couple of times. Maybe it's just because I'm more familiar with 0-based indexing, but I think it's not just that. In every case the 0-based indexing had fewer (+1)'s and (-1)'s than the 1-based code. For instance k%n returns a number in the range [0,n), or in Julia notation [0:n-1]. When you want to do array indexing with that, you need (k%n)+1 in Julia.

Btw, Matlab also uses 1-based indexing, no? I thought that Julia chose 1-based indexing to get brain backwards compatibility with Matlab.

Not just Matlab, but many technical computing languages use 1-based indexing. Julia was never intended to have any compatibility with Matlab.

It's just bikeshedding IMO. When I write C# code, array indices are like memory operation in my mind. When I go to Octave indices are like subscripts on sequences to me. Anyone who think it really makes a difference is being ridiculous.

I think zero based is "prettier" because of half open intervals, but it really doesn't matter.

I don't think it is bikeshedding IMO from a pure mathematics point of view. In set theory you begin with the empty set as zero, and the next integer is N+1. I find it interesting that the Julia devs have gone with APLs style of slicing, and did not move to 0-based indexing. Ken Iverson, who created APL and later the J programming language, chose 0-based indexing for J. APL has a system variable QUAD-IO that allows you to choose your indexing value of 0 or 1, which leads to many issues. Iverson's books on math (Arithmetic, Calculus, and the Concrete Math Companion (Knuth's book done in J)) all to demonstrate mathematics in J, all begin at Chapter 0. I think the choice of index in APL is a bad idea. I have run other people's code, and realized it doesn't work because they set their indexing to 1 vs. 0 for me. People and programmers can work around most things, but I think if you are a functional programmer, and mathematics is your thing, you are more comfortable with 0-based indexing. This of course does not address scientists, for whom pure mathematics is not at the forefront of whatever they are working on. I do think it leads to problems down the road for 1-based indexing.

Julia looks very interesting but honestly, exactly the 1-based indexing makes it feel really awkward and turns me off every time I think about using it somewhere.

Wouldn't it be possible to create array classes in Julia that all indexed zero-based? I would welcome a 'Julia0' fork :-)

While I agree that it's an annoyance, it's a very small annoyance. Since Julia has for loops and Julia ranges use the same 1-based convention you don't run into it often.

I feel that 1-based indexing has been the right choice, and I am aware that many do not like it. I really haven't run into off-by-one issues much, but this is probably more about how one is used to writing code. I like that you can iterate over a vector of size n, by just going from 1 to n.

Coming from a regular programming background, as opposed to a technical computing background, can certainly make this look alien. However, I think it starts feeling natural quickly. There are many other things in Julia that can take a lot longer to get used to.

It sucks for slicing though. In python (zero based), you can use 0:n to slice the entire array with a half-open interval. With one based, you have to either have a closed interval, so 1:n is the whole array (which sucks because what does an empty interval starting at 1 look like? 1:0?) Or you can have a half-open interval and index 1:n+1. I don't see what you saved yourself other than making things uglier.

For those who don't like one-based indexing, your problem is solved: there is now a package which provides two-based indexing https://github.com/simonster/TwoBasedIndexing.jl ;-)

Edit: For a history of 0-based indexing see http://exple.tive.org/blarg/2013/10/22/citation-needed/ My understanding is that with modern compilers, there's no need for 0-based indexing; you can use 0-base, 1-base, or 69-base indexing as you prefer. 0-base is applicable to people creating chips or writing hardware drivers. Most scientific programmers are not so closely tied to hardware; 1-based indexing is one of the things that helps make Julia more accessible to high-school programmers or PhD students who don't want to learn binary arithmetic.

> My understanding is that with modern compilers, there's no need for 0-based indexing; you can use 0-base, 1-base, or 69-base indexing as you prefer.

It's never been needed, Pascal had arbitrary array bounds (including negative) 50 years ago.

> 1-based indexing is one of the things that helps make Julia more accessible to high-school programmers or PhD students who don't want to learn binary arithmetic.

What does binary arithmetic have to do with anything? And have there been any study pertaining to the accessibility of 0-indexed versus 1-indexed arrays to high-scool programmers which would warrant such an assertion?

Zero based indexing counts offsets. Literally in C, you are saying "add n times sizeof(T) to the base pointer and dereference it". It rather implies arrays are contiguous in storage and lack any "holes". It's a crude but close-to-the-metal approach. C arrays are not general, they are one specific structure. An array could be implemented as a tree, but C doesn't have built-in support for that.

One based indexing counts, or labels boxes. Counting implies no "holes", but abandons any implication about how they are laid out in storage. Labelling allows treating numerical arrays as sparse hash tables, with as many "holes" as needed. It's a high level view, suited to mathy ways of thinking about data.

You can "label" by just agreeing that zero is in the set of counting numbers and labelling with those. Just because it's zero based doesn't mean it has to be an offset.

One-based is just wrong. I've never been in a zero-based language and wanted one-based, but I go the other way all the time. Arrays beginning with zero are isomorphic to the ring formed by Z modulo N, and you can use a bit mask for modulo with a power-of-two array size. It's not an "implementation detail" that arrays start with zero it's a "zero is important to mathematics" thing.

If I want to build a hash table, a power of two size for the underlying array is a nice performance optimization since you can use a bitmask instead of a modulo operation, and it makes a big performance difference, what does one-based give you other than some nebulous "it just _seems_ simpler to think about" kind of nonsense? I love how we expect people to grok the type system but we're afraid zero-based arrays will scare mathematicians off.

See: https://en.wikipedia.org/wiki/Modular_arithmetic#Integers_mo...

My impression is that you aim to produce a perfectly clean and elegant – perhaps in order to avoid a Python-3-type scenario. However, if that is what you are doing, I am afraid that Julia could become the GNU Hurd or programming languages. Most languages, at least those that have practical relevance, have design flaws. Heck, R is one giant mess of design flaws, and yet it is actually amazingly useful. Another example would be Perl. I think design flaws can be a sign that the creators of these languages are reasonable people who are capable of making pragmatic decisions (instead of just academic decisions). I’m clearly not an expert and I’m not up-to-date with what is going on in Julia but as a potential user I have the impression that we’re not moving closer to a point where Julia is ready for production use, and I’m worried that Julia will lose momentum while R is growing stronger and deeper roots. Personally, I really like Julia and would prefer if it would replace R but over time this is becoming less and less likely. The world is full with examples showing that inferior technologies can become dominant just because they had a head start. I think this article is relevant here: http://paulgraham.com/startuplessons.html

On the other hand, Rust spent a long time undergoing tumultuous upheaval, considerably more radical than this, i think, and yet it did eventually stabilise and produce a 1.0.

Oh yeah. Rust was really four different programming languages over a roughly eight year period. Always the same focus, but radically different ways of accomplishing it.

Personally, I would much rather see Julia undergo serious upheavals in an attempt to make an elegant language that works really well for scientific/HPC computing. Too many languages get trapped trying to provide backwards compatibility for everything and you end up with a lot of unintuitive cruft and quirks. If you want a language for "getting stuff done", use Python. But I find that I get a lot done with Julia in spite of constant change (in fact, I don't use Python at all).

Whether Julia replaces R has more to do with how R evolves over the next 5 years than Julia IMO. As far as "production use" I think that depends on whether what you're doing is well served by the existing libraries or not.

R as a language is not evolving at all and that is not a problem because the strength of the R system is its incredibly rich ecosystem. Julia cannot evolve a similar ecosystem as long as it doesn't provide stable foundations and for this reason it is completely uninteresting for most of the audience despite its superior design.

Yes and no; base R is extremely stable, but if you need to install dplyr, maggrittr, ggplot2, Rcpp and knitr/rmarkdown to reproduce a large fraction of the work that's been done in the last few years that seems somewhat equivalent to language evolution. But with rough edges, because some of the nice workflows that this opens up (say piping) are not necessarily in line with the way R wants to deal with objects.

In my experience, Julia is uninteresting to most statisticians because of the lack of mature libraries and because R is already good enough for most of the stuff they want to do. I don't think that relatively small language changes like the one discussed in the original issue are even on their radar, and I doubt that these changes are driving away people that would otherwise be maintaining packages.

Reply on news.ycombinator.com