Hacker News Reader

Comment by andy800 | Hacker News Reader

Why does every web site default to aggressively paginate their information? Pagination sucks, it's a waste of time and of clicks, and should be a last resort. Sure, when Google returns millions of search results, paginate. But:

For instance, if you have 40 entries and specify that there should be 10 items per page

40 entries??? Just show them all to me. My browser has a scrollbar, CTRL-F is much faster than your search box.

"but not everyone has a reliable fast connection" -- yes, which is a good reason to deliver more data per http request than breaking it up and requiring lots of slow requests.

"but the database load" -- half the queries, each returning 2x data, is almost always going to be easier on a RDBMS. If it's not then you probably need to rethink your schema.

View on HN

> Why does every web site default to aggressively paginate their information?

You get to see more ads while flipping through pages.

Timing metrics for loading smaller pages make marketing happy.

Timing metrics for time spent on the website make marketing happy.

Perhaps, for content-based sites. Not if you're a financial institution and visitors to your site are likely looking to find a specific transaction. Not if you are an insurance provider and visitors are trying to find a service provider. If you are a retailer selling stuff, you don't want browsers, you want buyers. I will concede that Amazon does pretty well and they paginate product listings, but I think they use a lot more intelligence to deliver high-value results than typical retailers, including very large ones.

> Why does every web site default to aggressively paginate their information?

Because data retrieval costs money and resources. Unbounded API responses are a susceptible to denial-of-service. Well-architected APIs will always cap the number of results they return per-call.

Nothing needs to be "unbounded." Set the cap at 100 or 200 records instead of 10-20. Better yet, plenty of results can be cached. A custom search might require a database query, but Newegg's initial listing of MicroSD cards, graphics cards, hard drives, etc, -- queries that are fulfilled identically hundreds of times per day -- can be cached.

At which point you're not actually arguing against pagination, you're suggesting a higher page size.

I agree, page sizes are usually far too short by default! Especially when I'm on a desktop computer and a fiber connection. I think people using phones and cellular connections probably appreciate short page sizes, so it would be nice if sites had different page sizes for different clients.

Do you perhaps conflate API level pagination with display pagination?

How many people actually read past entry five?

Your comment is half-way down the page. I read it.

It really depends on what kind of site we're talking about; there's loads of times I want to see loads of results. Sometimes I don't, but there's no harm is showing me more than I want, either.

I’ve seen instances where the issue is not database performance, but web server performance. Serializing a graphql response in graphql-ruby, for example, can easily take 100+ms and grows linearly with number of records in a request.

In the general case this isn’t a good excuse, there’s no reason it should take that long. But you don’t always get to pick your technology.

I'd guess that most people have written some bad query that only worked for small limits, so they started superstitiously making their limits very low.

And this kind of thinking is how you end up with 2mb per request.

It’ll be like Jira, pulling in literal megabytes of data and still slow.

No, relying on mega-frameworks and non-optimized graphic images are the express train to large request sizes. Don't forget video.

Indeed.com's search results are almost all text. Maybe a company avatar. No reason it can't provide 100+ job listings per page rather than 10-20.

I don't think the problem there is the pagination (or lack thereof). This page has 103 comments right now and is 204K (31K compressed).

HN is pretty minimalistic, but the point is that text data isn't really what makes requests very large. For reference, 2MB is enough to include an entire book and then some. All of Dune is about 1.7M uncompressed in epub format (and ~1.1M compressed as an epub file). HN threads, Jira discussions, etc. can get pretty large, but I have not seen any yet that have started to approach the size of Dune.

Images might add more data, but those can be loaded on-demand when scrolled into view pretty easily.

I don't think that is a bad thing, considering that a YouTube video in HD would be 500MB and people flick through them like web pages.

I agree with OP that if you are on a powerful machine with fast internet it would be better just to load a massive HTML document of list items.

Reply on news.ycombinator.com