Comment by haberman | Hacker News Reader

Comment by haberman | Hacker News Reader

Arguing that this cannot reasonably be done is unconvincing when well-designed languages like Lua pull it off.

The only thing about Lua's tables that consistently confuses people is the behavior of the length operator in the presence of "holes." The previous, somewhat confusing definition in Lua 5.1 (http://www.lua.org/manual/5.1/manual.html#2.5.5) has been replaced with:

    The length of a table t is only defined if the table is a
    sequence, that is, all its numeric keys comprise the set
    {1..n} for some integer n. In that case, n is its length.
    Note that a table like

     {10, 20, nil, 40}

    is not a sequence, because it has the key 4 but does
    not have the key 3. (So, there is no n such that the
    set {1..n} is equal to the set of numeric keys of
    that table.) Note, however, that non-numeric keys do
    not interfere with whether a table is a sequence.

    A program can modify the behavior of the length
    operator for any value but strings through the __len
    metamethod.

(http://www.lua.org/work/doc/manual.html#3.4.6)

Not only does Lua have the length problem, but array offsets are a matter of convention. Lua chooses 1 as the first element of an array instead of 0. But that is just convention. Your code could use 0 or -100. A confusing implementation of length and offsets by convention instead of using nearly universal array semantics from other languages is not "well-designed". It's a cheap hack to conflate everything into one table data structure.

> array offsets are a matter of convention. Lua chooses 1 as the first element of an array instead of 0. But that is just convention. Your code could use 0 or -100.

Not sure what your point is here. Lua arrays start at 1. If you disobey this rule, your table isn't really an array and the length operator will be undefined starting in Lua 5.2.

Sure you could disobey this rule and create an array that starts at -100. You can do the same in C:

    // Array whose indices start at -100:
    char *funky_array = (char*)malloc(n * sizeof(char)) + 100;

> It's a cheap hack to conflate everything into one table data structure.

You say "cheap hack" I say "brilliant optimization." Show me an equally-capable language that has a <100kb interpreter and performs as well as Lua. Lua fulfills its design goals brilliantly.

Perl 5 has $[ which lets you set the base for every array. eg. $[ = 1; would cause all arrays to be 1-based. Thankfully they've now deprecated this misfeature ...

http://perldoc.perl.org/perlvar.html#Deprecated-and-removed-...

I thought that was deprecated a long time ago. When I learned Perl in 2000 I remember I read not to use that "feature". Apparently it was just not recommended, for a very long time.

Certainly the advice has been don't use it for a very long time. At least since the 90s when I learned Perl. However it wasn't actually deprecated until last year (Perl 5.12). There were a sequence of steps leading up to full deprecation which you can read about in the link I provided above. Maybe some people were actually using it?!?

I'm not convinced Lua quite pulls it off. The combination of array and hash is awesome and I love it, but as a practical matter, all my tables are either arrays or hashes (objects), and I treat them as such. They aren't really unified, and this difference permeates everywhere, from implementation details, to the C api, to high level functions like pairs vs ipairs.

The blog post was trying speculate about slicing an array to a hash, or iterating backwards from "-1" in a hash, or something. Whatever he wanted (not quite sure), you can't do that in Lua. I don't know why you'd want to either. So Lua doesn't pull it off, but there's no reason to.

> The blog post was trying speculate about slicing an array to a hash, or iterating backwards from "-1" in a hash, or something.

It was mentioning features which don't really make sense on associative arrays, or are not compatible with mixed usage:

* Some features (e.g. sorting values) simply can't behave the same between arrays and associative arrays (on an array it will change the value's index, but it can't change the value's key on an associative array)

* Splicing (not slicing, though that would probably work as well) hardly makes sense on an associative array, you just don't need it (splicing is the replacement of an indexed section of an array by an other array — which depending on the language may be of a different size, including empty): you can just set the keys instead (e.g. combine with an other associative array)

* The final issue is with indexes, including "interesting" indexes (direct and in slices/splices) such as indexing arrays from the end (a very nice feature of some languages: Python and Ruby share it with perl) using negative numbers. You just can't do that if you mix things up, so you have to rely on calls like `val = a[length(a) - 2]` instead of just `val = a[-2]` (note that there are languages out there which do not conflate arrays and associative arrays and still don't implement this feature, and a restricted class of languages on which you can actually have negative indexes on arrays)

So Lua is able to pull it off, but there's something that consistently confuses people? Forgive me, but hose sound contradictory.

There's only one thing that consistently confuses people, it has nothing to do with any of the objections raised in the article, and it's being fixed to be less confusing.

So Lua has negative indexes?

It has to do with "holes" in the array. If you have a Lua array like: { 1 , 2 , 3 , nil , 5 }. That "nil" causes problems. I, personally, haven't had an issue with this, but it does come up regularly on the Lua mailing list.

It may be better to think of it as a key/value table (a "dict" or "hash") that recognizes when it's being used as an array and optimizes its internal storage accordingly. It always has dict/hash operations, but indices of 1..n are stored more efficiently, and a couple functions reflect this common & algorithmically significant case: the length operator returns the n of 1..n, and "ipairs" iterates over the {1, value}..{n, value} pairs. (The un-ordered {key,value} iterator is "pairs".)

Typically, it's clear upfront whether you're using it as an array or as a table; there isn't much overlap. I understand it being confusing in the beginning, but it's really not a big problem, it's just Lua's version of "oh god, the parens/significant whitespace/etc.". (That and indexing from 1. Not a fan of that personally, but whatever, Lua is handy.)

So yes - you can use negative numbers as indices. Or strings. Or other tables, or arbitrary C pointers. Arrays are just handled better.

Except that it was mentioned, at the very least tangentially, in the article:

what to do when, for example, someone creates an entry in an array by giving it an ordinal position that doesn't exist. Do we create an array of suitable length and fill it with bits of emptiness in order to maintain the illusion that this array is ordinal?

And that sequential tables are 1-indexed, not 0-indexed like nearly every other language on the planet.

That was intended to make it less confusing when non-programmers use it as a configuration language. It's a bit annoying, but the language is consistent about it: if all you want is a tiny library for JSON-like config files or data dumps, it cleanly scales down. If you want a powerful, modern hacking language, it's there, but the advanced features (tail-call optimization, co-routines, etc.) never get in the way.

Reply on news.ycombinator.com