And I'd bet the reason why it's still there (I left Search in 2014) is because < 0.1% of users ever hit the next page. Everybody else just refines their query to a different search. It's a holdover from when search engines were bad (i.e. around 1998) and you had to go through 10 pages of results to get the one you were looking for. As a result, Google expends approximately zero engineering effort on pages 2-20 of the results - I know that in the 4 visual redesigns I worked on, we didn't touch them once. It wouldn't surprise me if the response to flack on this is to just get rid of all pages other than the first one - it avoids the issue entirely and wouldn't affect 99.9% of users.
The technical reason for this behavior, as others have remarked below, is pagination. Ranking across the full result set is a very complex calculation, and it can depend on some factors that are basically random (eg. timeouts and failures in backend servers). It'd make pagination basically useless if the same results you already went through show up on a later page because the ranking is different. This requires that the full result set be cached. You can cache 400-1000 results for each of the queries that the 0.1% of users who actually hit "Next" care about, but you'd have a big issue caching 2 billion results for each of those queries.