The trouble with 'ntile()' | Hacker News Reader

The trouble with 'ntile()'

blog.djnavarro.net

26 pointsluu3 comments

Tl;dr: The trouble with ntile seems to be that, when the author of the post imagined what it did based solely on its name and she imagined wrong, she ran with her imaginary version way too long before checking the documentation, which was very clear about its behavior:

> Unlike other ranking functions, ntile() ignores ties: it will create evenly sized buckets even if the same value of x ends up in different buckets.

I’m pretty sure NTILE is just a generally cursed function. SQL NTILE also requires everything to be loaded in memory because of the odd rule that larger buckets precede smaller buckets, so it’s unusable on anything decently sized.

Not being able to specify how ties are handled (and choosing, as far as I can tell, a fairly useless definition) fits the bill.

If it is "cursed" for a function to do what it is clearly documented to do, instead of what someone mistakenly imagines it to do, then what function isn't cursed?

> Not being able to specify how ties are handled (and choosing, as far as I can tell, a fairly useless definition) fits the bill.

Actually, you can completely specify how ties are handled—and should, in any study designed to be repeatable. The docs for dplyr::ntile tell us that:

> To rank by multiple columns at once, supply a data frame.

So, repeatable, completely specified tiebreaking is as easy as adding a tiebreaker column to the dataset, using whatever strategy makes sense for your study. For example, if we wanted random tiebreaking using R's built-in `runif`, all it takes is one extra line of code:

    data_table |>
      dplyr::mutate(
        tie_breaker = runif(dplyr::n()),
        ntile_bin = dplyr::ntile(tibble::tibble(partitioning_key, tie_breaker), n = 4))

Almost 100% of the original author's problems could have been avoided by just reading the docs.

Reply on news.ycombinator.com