Hacker News Reader

Comment by goosejuice | Hacker News Reader

Congrats! Been running a startup off only liveview for about a year as a solo dev and it's been wonderful. Appreciate the work you all do.

Selfish demo idea: Bi-directional cursor based infinite pagination with largish datasets with state managed in the url and streaming updates that change the order of the results. Like some kind of soft realtime leaderboard.

With long render times (morphdom bench on large sets isn't great as far as I can tell) it's hard to escape the jank. Particularly on slow connections.

View on HN

Will keep this in mind! My ElixirConfEU keynote just used limit/offset pagination, but shows how easy it is to do bidirectional infinite pagination. Streams allows you to keep a virtualized infinite list in the DOM without keeping the giant collection around on either client or server, example:

https://www.youtube.com/watch?v=FADQAnq0RpA&t=2322s

This demo doesn't update the URL to keep state (but push_patch would make that trivial). I'll think about more complex examples for cursor pagination, but the solution to massive collections (thousands of entries) is going to be stream + limit

Thanks! That is the approach I took, though I believe reordering rows requires a reset.

So in that demo, I'm curious how smooth the scroll would be when adding latency via the debug tools in the browser console. Even with some padding and fiddling with limi ts I've had trouble getting it to be an enjoyable experience. Just bouncy if that makes sense. More frustrating on mobile than desktop. With localhost latency it's great.

I'm thinking it may just need a hook to preload rows or something of that nature to provide space for the render. Though my CSS could very well be wrong.

flop_phoenix is great. If it lacks anything maybe it'd be great to just enhance it instead of building anything new. The author also has cursor based functionality

I highly recommend checking out [Flop](https://hexdocs.pm/flop/readme.html) and its Phoenix.Component [Flop Phoenix](https://hexdocs.pm/flop_phoenix/readme.html).

Its extremely easy to customize and build a reusable paginated table component

Second that

The flop is a good idea. Love the flop

How much state is it safe to store in the URL these days?

Still not much, realistically 4096 bytes or less.

Browsers aren’t as much the issue as they’ve been in the past, but I’ve hit snags with proxies, old servers, etc.

How does pagination in urls work nowadays? You'd need ~3 bytes to index a reasonable number of pages naively, no?

But curious what current art is re: performance optimizations between frontend and backend. Or is it simply page indices?

If you do limit/offset on database side, page number is enough. Though this doesn't work well for bigger page numbers. There's other ways to do pagination, e.g. with "cursors", where cursor is simply id of last record on previous page. SQL query is very efficient, but jumping to page X is impossible. In this scenario storing cursors for past pages is needed

There are libraries for Ecto that help with this.

https://github.com/duffelhq/paginator

Careful that this library last I used it (2020 or so) used a particularly insecure encoding of the cursor that basically allows remote execution. Not sure if they ever addressed it.

Here's the fork I created at the time to work around some of these issues: https://github.com/1player/paginator

This looks like it was fixed

* https://github.com/duffelhq/paginator/blob/main/lib/paginato...

* https://github.com/duffelhq/paginator/blob/main/test/paginat...

Thanks to thread for the informative responses! (And the useful README on your fork/the upstream)

I try to assume someone's thought of better than the best I could, or at least learned the hard way what edge cases need to be handled.

Thanks for pointing that out.

Reply on news.ycombinator.com