Hacker News Reader

Comment by andrewmcwatters | Hacker News Reader

It is inevitable that people complain about Lua's 1-based "tables" which serve as both arrays and hash maps.

I usually encourage people to remember the practical origins of 0-based arrays when thinking in terms of pointers, where 0-based counting is useful in terms of talking about offsets from a point in memory, as contrasted to 1-based counting where you are primarily talking about quantities of collections.

Lua is a great language. My only wish is that it was easier to do multithreaded work in it. Unfortunately, this is not the case without some significant architectural forethought.

View on HN

The reason why 0 makes sense for pointers isn't due to some arcane implementation detail of programming but because of how one conceptualizes ranges and distance and pointers just so happen to be a place where that comes up very directly in programming. Any time you attempt to develop an algorithm that needs to think about multiple ranges of a collection at the same time or needs to scale ranges through a collection, 1-based indexing causes you to have to throw in a bunch of +/-1 compensation factors to prevent you from double-counting or scaling past an edge. Life is just too short to have to constantly debug the issues that come from 1-based indexing when developing complex algorithms, even if it means beginners feel a bit confused by "counting from 0".

No, it actually does originate from 1960's era programming practices. Not to say there weren't also 1-based or N-based numbering, but you're historically misled here.

You could also say the same about counting collections. No regular person counts 0... 1... 2... 3... and ends counting N-1. It, too, is a workaround.

I don't think saurik is talking about history, so you're talking past each other in that sense.

I think so, too, but my point is that it makes sense for other historical reasons—not the one the poster mentions. Those decisions weren't made due to mathematics.

Agree. I came here to say that 0-based is more sensible, but as I thought through my arguments, I realised that I’m wrong. I do most of my coding in TS, Go and plpgsql. PL is 1 based, but the only time I find it frustrating is when translating between language boundaries. The rest of the time it’s just convention (eg, using <= instead of < in loops).

But I realised that I frequently want to get the last element in an array, and of course it’s easier with 1-based arrays.

So ultimately I don’t think it matters, except that most people are used to 0-based which can cause cognitive friction.

We aren't talking about counting. We are talking about indexing.

Most programs don't need numeric indexing. They can either treat indices as opaque keys or they can use a foreach loop to avoid touching indices at all (or friends like map, filter, zip, and so forth). Even in an old language like C the only place indices usually matter is in the first line of a for loop. But sometimes you really do need numeric indices, and when you do zero-based indices are almost always cleanest.

Just think of how gnarly a flat 2D array would be if you had to use one-based indices. Gross!

I don't think 0-based was ever as common a practice as it is today, especially in the 60s. In the beginning there were two camps fighting which system was to be preferred and there were many more 1-based or agnostic languages (like the whole Pascal family). "Numerical Recipes" a pretty popular book for some time even came in two editions, a 0-based and a 1-based version. I think the victory of C had a big influence on 0-based winning.

Personally I find it very sad that the agnostic approach of Pascal did not prevail. Let the programmer decide what they think fits the situation best and let the compiler deal with the technicalities. This, and I also miss range types... for like 40 years now...

Surely everything was 0-based initially, when everything was written in assembly? Are there any 1-based ISAs?

I wouldn't call assembly 0-based. Sure, the zero flag is convenient for branching on zero when counting backwards but what makes assembly 0-based apart from that? I'd say having a pointer to the first element of an array is pure convention and you could as easily point it to the imaginary element before, similar to how the program counter always points to the next imaginary instruction on most architectures.

The offset addressing modes all start from zero. You could hack around it in some places by storing the pointer to an element before but that's clearly going against how it is intended to be used.

For example if you have an array that you cast between different sized types (e.g. uint64 to uint8) then you have to change the pointer value!

> Personally I find it very sad that the agnostic approach of Pascal did not prevail. Let the programmer decide what they think fits the situation best and let the compiler deal with the technicalities. This, and I also miss range types... for like 40 years now...

I am with you there. However, Pascal, a toy Algol, was designed to be a language for a compiler course. It took its ideas like "where does an array start" from other Algol implementations and added a broken type system. Pascal gets the love now, as witnessed by the Lua designers referring to Pascal and not to Algol.

If the 1st element is [0], something's conceptually wrong.

When did the 21st century begin?

[0] just means the address of the element is ptr+0. The first element is at the address from a memory addressing perspective it makes sense that it would be at [0]

Why care about the memory address for higher level languages like Lua?

The current element is at distance 0 from iter[].

You are right, but the thing that is wrong is that the word "first" is associated with the number 1!

Think about it. Natural language could easily have associated the initial ordinal with 0. Let's call it the Nilth (zeroth is really hard to say).

The nilth element is element 0, the first element is element 1, etc.

The winner of a race is in nilth place. Silver goes to the person who came first. Bronze so second.

You walk in to a building on the nilth floor, climb the stairs to the first floor (that might sound weird to Americans but it's how it works in other parts of the world).

This is probably all sounding crazy to you but it's way more elegant and self-consistent.

Which century is the year 2023 in again?

Some of us Americans know our elevator numbers are wrong — zero / nilth is the correct ground floor by far! — but that doesn’t mean when you pick up the first apple in a bowl of apples you should rename it to nilth. Counting order is different from Indexing order. One is the order of objects in a list and the other is the offset (2nd floor is two floors up from zero). They are just different things and that is why they sound right in different contexts!

(It’s 2023, and we both are right)

Correct, counting and indexing are different things. But the "first apple" is an index not a count.

> 1-based indexing causes you to have to throw in a bunch of +/-1 compensation factors to prevent you from double-counting or scaling past an edge.

Way back then, I started with Algol 68, moved on to Turbo Pascal, and finally Fortran (I was young and needed the money). My vectors and arrays started at 1, as was the default, and I do not even know what you allude to. The first element has the index 1. Since there is no zeroth element - neither in language nor in thinking -, starting at 0 is what causes problems. You always have to use a corrective term in your head - the element five elements after the third one has index 7.

Yes, whatever.

My experience is that the +-1s come up less often in 1-based languages if you're writing high-level code, and when they do they're more intuitive to catch, even after years and years of coding 0-based.

The problem with 1-based arrays is that it makes the offset and location be different. For example, we want to place an item into a simple circular buffer with zero-based indexing:

arr[idx mod capacity] = 123

Now with 1-based indexing:

arr[(idx-1) mod capacity + 1] = 123

This is absolutely makes it a PITA if you're working with things like bitmaps, blending, or sound mixing. If you only need a language for high-level scripting, then it indeed is not such a big deal.

Lua’s arrays can start at index 0 if you like, there’s no real reason why the first line wouldn’t work.

Starting at 1 is a convention:

array = {“hello”, “world”}

print(array[1]) —- displays “hello”

array[0] = “why”

array[2] = “there”

for i=0,2 do print(array[i]) end —- “why” “hello” “there”

It's cute that you can do this, but it's not a practical solution, so I don't really know why people keep bring this up.

As shown by your example, arrays constructed the native way will start at 1. So you're going to need to make your own custom constructor which means additional overhead.

Also, all built-in functions that expects 1-based arrays will completely break:

    for _, value in ipairs({[0]="foo", [1]="bar"}) do print(value) end

This will print "bar"

And even if you still insist on pushing through all of this friction by handrolling your own stdlib, everything is out of the window once you interact with any external library that you don't control.

There's a very good reason why no one is using this trick. Anything other than 1-based indexing is just undefined behavior.

There are different uses for `for’, and you’re supposed to use `for i=0, n do’ for the use case you’re describing.

People absolutely do this, and you have to for things like using LuaJIT’s FFI and doing work across C.

Lua tables support do 0 as an index. You can also use negative numbers or floating point numbers, which can be convenient for some applications.

One issue with using them that way is that some Lua idioms don't work as expected. For example, "for i,v in ipairs(t) do ... end" will start iteration with i=1 and stop when it reaches an index that isn't present. Also the length operator (#) won't work as expected.

What do you get when you get the 0-index? Does x[0]=x[1]?

Just reading the comments here, x seems to always be a hashmap. It is implied that there's no fundamental array type in the language. So no, x[0] and x[1] do not refer to the same variable, although they certainly could compare the same depending on their specific values.

Works fine in Fortran, R, and Julia. People do pretty complex stuff in these languages and two of them are compiled.

It's not an insurmountable barrier, for sure. Just something that makes it very annoying in a lot of code. While not really making it easier anywhere.

I've been working on a PR to document and make atomic pointer interface public in Julia. It's really not that hard to handle it under the hood so that array indexing and translates to LLVM and C zero indexing.

Weird then that scientific computing languages prefer 1 based arrays. Maybe can show how it's very annoying in lots of Fortran or Julia code?

Python is as much a "scientific computing language" as anything, and it's zero-based.

You haven’t shown how it’s “very annoying” in R, Fortran, and Julia.

Not really, the core language is general purpose. It's Python's scientific libraries and ecosystem which became popular.

> Weird then that scientific computing languages prefer 1 based arrays.

Not really when the code is routinely transcribed from papers, and the ages-old convention throughout mathematics is one-based indexing, both bounds of the interval included. (With the notable exception of Euler’s beta and gamma functions.) Of course, in a paper intended for a human reader the occasional off-by-one error is much less dire than in code, so the author is freer to use a more error-prone convention.

I once had to translate from a one-based paper to zero-based Python code (with the added bonus of different discrete Fourier conventions *shudder*), and it’s remarkably annoying even if the resulting code is more obviously correct than the paper’s original formulas.

It is potentially odd (or even) but nothing compared to everything else when it comes to programming oddities or quirks.

What is the first item in this collection? Oh that would be the one in the box marked zero unless it should be in the one marked ... one!

You mention a circular buffer as an outlier but you simply have to decide where it (the buffer) started off life - 0 or 1 or something else. Make sure that when it is defined, everyone agrees on offsets and off you trot.

No it isn't particularly easy but then bitmaps and blending and sound mixing involve some very involved concepts that are way more complicated than sorting array bounds and worrying about where you are on the Magic Roundabout.

> You mention a circular buffer as an outlier but you simply have to decide where it (the buffer) started off life - 0 or 1 or something else.

No. A circular buffer is just a simple example where you need to go between offsets and locations.

The same theme happens in many similar cases, like directly manipulating bitmaps. E.g. with zero-based indexing into an RGB bitmap:

  bitmap[y*width*3 + x*3] = 1

vs. 1-based indexes:

  bitmap[(y-1)*width*3 + (x-1)*3 + 1] = 1

The point here is that with the Lua code the CPU is doing some extra work because mod is 0 based because of mathematics and Lua is 1 based by design.

However mod itself is doing extra work compared to incrementing a counter by one.

This is pseudocode because I don't use Lua much and I don't have the time to check the syntax now, but it should give the general idea

  class CircBuf(size)
    buf = new Array(size)
    i = 1
    add(value)
      buf[i] = value
      if i == size
        i = 1
      else
        i = i + 1

The relative efficiency boils down to a comparison+increment vs mod on the particular CPU the code is running on. Mod should be a division, unless size is a power of two.

Yes that is annoying the first week, then you get used to it.

If the biggest complaint about Lua is 1-based tables, I’d say that’s a compliment to the language indeed.

Dijkstra had an opinion on this (and many other things). 0-based ftw.

https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF

Bit short-sighted of him to only consider positive array indices IMO. Lua allows them to be negative.

Lua also allows floating point indices. In fact, any value can be an index into a table. That means any number, string, True/False, function reference, thread (coroutine) reference, table reference, "userdata" reference (usually a pointer to a C object), or nil can be an index.

Correction, any value except nil can be an index into a table. https://www.lua.org/manual/5.3/manual.html#2.1

Dijkstra conveniently ignored the other situations where 1-based works out better than 0-based. For example, getting the least element of the array, or iterating backwards over the array.

This reminds me of that somewhat famous observation about the bright scientists working to increase clicks at Google/FB instead of finding cures for diseases and such.

So much brain power to defend something that existed a long time ago and was adopted as some kind of law of the Universe when there is clearly no good reason to be this way today other than "it's just how it is" and every new programmer has to pass for a quick phase where the 1-based it beaten out of his mind.

I'm not going to complain about the indexing but I am going to complain about tables. The fact that you need at least one element to be in a table to trigger heuristics to serialize it properly makes Lua uniquely bad for use in polyglot distributed systems.

I've specifically run into this with OpenResty and JSON processing...

I would avoid trying to serialize tables in polyglot systems since it sounds like a specialized data structure whose counterparts don't really exist in other languages anyway.

It's a specialized data structure that the whole Lua ecosystem drives you to use as your primary collection type. They're blessed by the language, every library function will use will take and return them, and your deserializer will produce them, whether you like it or not.

Ultimately, you can use different data structures in Lua... but you'll have to write reams of boilerplate to do it. That's why I'm saying that (unfortunately) it's Lua itself that's a bad fit, not just tables.

I think it goes well with the loop syntax. In contrast to C/C++ where you do

    for(i=0; i<n; i++)

In Lua you do:

    for i=1..n

On the Lua case, both the beginning and end of the loop are inclusive.

If you have to iterate over n elements, the first syntax is optimized for 0 to n-1, while the second is optimized for the other case. See:

    for(i=1; i<=n; i++) // <n becomes <=

    for i=0..(n-1) -- ugh

The difference between 1-based and 0-based indices is irreverent enough, that’s the reason it feels uncomfortable for choosing to use something that is different. It is like little-endian vs big-endian, but you never see people choosing to use big-endian now

Reply on news.ycombinator.com