I'm positive this is usually due to convention, not a conscious design decision. In other words: websites give 20/50/100 row pagination options because other websites give 20/50/100 row pagination options.

There is no meaningful implementation difference between providing 20/50/100 options and 20/50/100/500 options, with 20 as the default in either case.

I would be very surprised if this decision is even discussed for more than 1 minute on most projects, let alone if any user research is done. But I'm happy to be proved wrong!