Array.js: A better array for the browser and Node.js | Hacker News Reader

Array.js: A better array for the browser and Node.js

github.com

There's no doubt this is a nice little library. There's also no doubt that I won't use it. Why? Because all the neat stuff in this library is available with Underscore, and Underscore works with plain vanilla arrays.

It might be possible for a library like this to really catch on so much that a) other people use it, and b) even when they don't you get so much power you don't mind typing array(someOtherRealArray) all the time. JodaTime is like that.

But the thing that JodaTime has that Array.js doesn't is an enemy. JodaTime hates the built in Date, Calendar, etc classes in the JDK. It hates them and replaces them with extreme prejudice, and anyone forced to work with those APIs does not weep.

Array.js does not have an enemy, it has a friend it wants to help be a little better. It's a nice library, a helpful library. And that's why it will soon be forgotten, I'm afraid.

(disclosure, I work on Underscore)

I think this is a pretty neat approach. It would be fun to play around with a version of Underscore that implemented a version of the idea. I make take a crack at it...

Creating a new datatype would pretty seriously go against Underscore's implicit philosophy of being a set of functions you bring to bear on ordinary data structures. In other words, you can try, but I suspect your pull request be voted down fairly decisively.

EDIT: apparently you are the creator of Underscore. Which makes me feel sort of silly for saying at least the second part. I think the first part stands, however.

Probably, but I'm thinking that it might be an approach worth taking a look at as an alternative to the current:

    _.chain(array).map(func).filter(func).reduce(func).value()

Taking a look at returning an array-like (or object-like, depending on what you pass in), for that same use case:

    _(array).map(func).filter(func).reduce(func)

... could be nice. The devil's in the details of interoperability with other libraries, as other folks note in this thread.

So in order to save one call to chain() you'd use a non-standard object over a native array? What would underscore methods return, then? Perhaps it could do type detection and return what it gets, but then you have the issue that the executing code needs to be aware of that too - which means a type check at the caller before invoking the underscore expression. Needless to say, I think that's a lot of added complexity for not a lot of benefit.

Heck, if you really wanted to adapt something from Array.js you could do the naive (but I think correct) thing which is to add a flag to all underscore methods indicating whether or not the method should return an "Underscored" object or the basic data type. This could even be an initialization parameter for Underscore.

What does this buy you? I can only think of two things. First is they get a more object-oriented feel to their code.

    var foo = _.filter([1,2,3], func, true);
    // later on...
    foo.filter(func2);

I guess the first question I have is, what does line 3 do? Are we mutating foo? Or returning something new? In which case we've managed to save precisely one character

     _.filter(foo, func2);

Even worse, we're saving the wrong character. One undersung quality of having one global object per major library is that, in the code, you get a big contextual clue about what's going on from the way it's invoked. Contrast these two lines:

    foo.filter(func2);
    _.filter(foo, func2);

Try to imagine that you're encountering these lines deep in the bowels of the code. Which one is easier to understand?

The second thing you get is a (what I think is) esoteric benefit of being able to "ship" underscore functionality with the data to far away places. This is only an issue in full require()-controlled programs where there might be some communication between closures, and one side wants to use underscore in that context, but not add a dependency. Personally, I think being able to do that is just asking for trouble, as, again, your calling code maintainers are going to be totally befuddled by running into objects doing things that no other objects in that side are doing - or worse, doing something that looks familiar, but isn't.

You're saving at least two characters there ;)

I think sensible semantic naming still helps in this circumstance.

    yummyBars = candyBars.filter(function(bar){ return bar.yummy; });

makes it clearer that the thing you're getting back is like the thing you had previously.

One place this could be useful is in Backbone.Collection where the array-returning mixins from underscore all hand back bare arrays, on which devs must then switch to _ prefixed functions (although you're right that this does keep a clear conceptual separation between collections and other arrays).

I get a warm and fuzzy when I know what that method does, that it's not some random filter that could mean something totally different. (And yes, this does mean that my code is not very object oriented at all!)

That's okay! Use the right tool in the right place. And agreed, I like knowing where a function comes from.

I'm not totally convinced by the move to 100% functionalist constructs (there are some places where OO is nice), but it's a nice stylistic practice.

It has one important downside tho - because you can't chain calls, your code ends up much less readable: It needs to be read from the inside-out, and the functions you're applying on the array are in reverse order compared to the order of execution. Its also harder to to see which arguments go to which function.

Compare

    _(array).map(func).filter(func).reduce(func)

and:

   _.reduce(_.filter(_.map(array, func), func), func)

> The devil's in the details of interoperability with other libraries, as other folks note in this thread.

Indeed. I should have also mentioned that editing angular.js to work is fairly straightforward (~5-10 LOC), and this is probably fairly typical. So if you are willing to patch this isn't necessarily a blocker, and done right could probably be submitted as a reasonable merge request.

This is where I point out that the idea of jashkenas getting a pull request rejected from Underscore would be pretty funny.

Not that it couldn't happen, of course, I don't know if he still is in charge or not. But it'd be amusing.

Yeah, as far as i'm concerned, jashkenas is still BDFL for DocumentCloud's major libraries.

I've taken over maintenance of the main DocumentCloud platform and document viewer repos, as well as docsplit.

And although I've been shirking on jammit, but I intend to cut a new version of the gem soon.

jashkenas is being humble. He's actually the creator of underscore (and backbone, and coffeescript, and docco...).

The way I did it was to write lots of functions that take e.g. an array (the same approach works for other things) as their first argument, then a function that I called unbind which converts a function of x arguments into a function of (x-1) arguments where the first argument is replaced with the this pointer. Then you can have a function ('bless') that copies all of those 'unbound' functions onto a real data object or prototype. That way you get the best of both worlds and it's up to the user as to what they bless.

The other aspect to this of course is that using es6 proxies, the library could have been made to behave exactly like the native array even without using one.

I would love to see underscore integrate this style of API.

This involves a lot "String Programming" which makes code hard to test and debug.

user.max("age"); looks sexy but I'm sure type of error you get from user.max(func)l is much better and more clear

I do like how the `select` can take a string, "x > 30", vs a function(x){ return x > 30}

It would be be nice if Underscore had a bit more sugar like that. But, I also know how problematic it can be to properly parse or evaluate a string into code.

--edit--

it is using the to-function library to do this: https://github.com/component/to-function

Passing it in as a string just feels dirty to me for some reason.

It feels dirty to me too, because of the possible "filter injections", similar to the SQL-injection flaws.

It's also likely to be inherently hard to optimize; aka slow and will stay slow. I thinks it's a generally bad idea. It also undermines what limited tooling there is for JS; and it exposes Yet Another set of semantics - what exactly does that query do, how is it translated, what are the corner cases etc.

Very Bad idea. (And for what - avoiding a few characters that are so common as to be trivially skimmable and very gzippable?)

What do you think about jquery selectors? Do the same objections apply?

I think they do too. I used to fall-back to vanilla DOM queries sometimes when I started using jQuery, because it didn't feel quite right, and I always worried about the performance hit... but I ended up always using CSS selectors, unless there's a good reason not to.

I think the main difference is that the value proposition of CSS selectors is just better than of the string-function syntax, so its worth it.

CSS selectors also give me this bad feeling when used in javascript code.

If you care about minor syntactical details like that, you should already be using coffeescript:

    select((o) -> o.x > 30)

Seriously, it's great. :)

Didn't understand all the JodaTime references, but as far as your main point, this library is doing the same things that underscore is doing - wrapping an array. Consider: _(['apple', 'orange', 'pear']).map(fn);

You could just as easily approach this lib the same way: array(['apple', 'orange', 'pear']).map(fn);

The only difference with this library and other arrays/client-side collections I've come across is that there's less indirection, which I find less confusing, ie. am I working with a collection object or am I working with an array?

Of course there are caveats (things to consider) when using this lib which I outlined in the docs.

I agree with you here; I usually have underscored included in both the server and browser for my projects. Events seem like it could be useful. Sure Backbone gives you events on models and collections, but I don't always include Backbone models in my node apps.

I think having events and query is overkill and is no longer an array. You are essentially making a database at this point but trying to pass it off as a basic array to replace the lightweight builtin Array. I would even shy away from calling it array since it really does very opinionated things.

I'm currently using both Knockout observable arrays [1] and Underscore array helpers [2] in a project, and the Array.js library looks like it mashes both of these concepts together quite nicely.

[1] http://knockoutjs.com/documentation/observableArrays.html

[2] http://underscorejs.org/#arrays

My first thought was "overriding the default Array object is dumb". But it turns out Array.js doesn't overwrite JavaScript's built-in Array objects. Instead it's an array-esque Object:

    var arr = array([1, 2, 3, 4]);   // An Array.js "array"
    var arr2 = [];  // A real JavaScript array

It's definitely too heavy for most array use cases. Sometimes all you need is a simple list.

Having worked on a similar project for clojurescript (1), I've found that the main challenge to using this "array spoofing" technique widely is interop with other libraries. For example, angular.js internally has inconsistent ways of validating an object is an array, sometimes bottoming out in Object.toString and other times instanceof Array. Many libraries behave similarly, and so array.js objects will not behave as expected if handed off.

On the other hand, interop with D3.js has been ideal as it generally assumes that any non function parameter is an array, so there I"ve had more success and would expect array.js to similarly work well.

(1) https://github.com/dribnet/strokes

Why do they use strings as function using to-function (https://github.com/component/to-function). Like .select("age>10").map("adress.streetname");

While to-function is cool and your code is shorter and more readable. You must be aware that you miss

- static code analysis such as synax checking - compile time optimizations (because the AST is missing) - static code analysis tools - code coverage checking tools

I would recommend using real functions, or using clojurescript or coffeescript in order to get shorter code.

This is really nice, and useful, I would love to see more like this, but wonder if there isn't some room to join forces between the best JS libraries out there and try to create a common utility belt compilation (e.g. work to avoid conflicts, and namespace collisions, joint testing, shared convention and merge efforts on similar things) e.g. I would love to have the JavaScript equivalent of Apache commons / Guava (previously Google collections).

I mean, if there is one thing I do like about Java (when I can't use Scala for some reason), is those utility belt libraries (and JodaTime of course). Whenever I hit a wall trying to do something in plain old Java, Guava / Apache Commons seems to read my mind and have a data structure, or a utility to the rescue.

Do you think it's a good thing to strive for? Do you see it happening?

if you look at the code, it's based on TJ Holowaychuk's component: https://github.com/component/component. there's i believe 600 "utilities" you can pick and choose from: https://github.com/component/component/wiki/Components. since it's CommonJS, you won't have any namespace collisions or conflicts.

i don't know how this relates to Apache commons, JodaTime, or Guava however, but i just wanted to add a little context.

For reference: https://developers.google.com/maps/documentation/javascript/...

Pretty much the same functionality, done three years ago.

Interesting. Though I wouldn't want to include the entire Google Maps API just to get a nice array helper class.

It's a nice little library but it looks like the meat of it is in Enumerable which is pretty nice but it's really slow when working with larger arrays.

If you don't need the events then lo_dash/underscore is probably the better tool.

... and you dont want something fired everytime you map().filter().reject().map().filter().some()... which might lead to nothing but bad performances ,especially on huge collections.

Is that a Schwartzian transform hiding in the sample code?

I've been working on something that does something like this for 2 years ... it's a busy field: https://github.com/kristopolous/db.js

lines like this:

   DB()
    .insert(what)
    .find( DB(".contentDetails.regionRestriction.blocked.indexOf('US') > -1") )
    .select('id')
    .each(Timeline.remove);

Better? If its using object properties then beware there is a very significant performance hit looping over object properties vs looping through a native array. Array looping is 5x faster in V8 trunk with a simple 10 element benchmark.

I look at this and think, again, that JS really needs a way to override attribute and indexing operations, like Python's __getitem__ and __getattr__.

There's not many things I miss after switching to CoffeeScript from Python, but this is one of them.

Also, I really dislike the custom string parsing implied by things like: .select('age > 20').

This is a place where CoffeeScript or LiveScript really shine. In LiveScript it would look like:

users.select (.age > 20)

This is coming in ES6, with Proxies.

looks nice, but why doesn't it just extend the core array type? (I mean with a "class", not by adding things to Array.prototype). That way things like Array.isArray() still work. Am I missing something?

This is also similar to the Backbone.js Collections type which inherits functional-programming methods from Underscore.

The main difference seems to be that Backbone expects objects as members of the collection.

I think there are more use cases for similar tool but based on Set instead of an Array. Otherwise worth noting

I was just thinking yesterday that something like this would make my life so much easier. You read my mind!

Reply on news.ycombinator.com