I found out the main reason why passwords are limited length even when using modern hashing techniques.
The problem is caused by the way multi-round password hashing algorithms are designed. Specifically, in each round, the full contents of the password is concatenated with the previous round's hash result as the input to the next round. This means that the processing complexity of password hashing is proportional to `number of rounds * password length`. Because the implementation has to tune the number of rounds to their authentication server's performance, and because the tuning is performed on "normal length" passwords, this opens the authentication servers up to a DOS attack by overwhelming them with very long passwords which is only a problem because password hashing has quadratic complexity thanks to this design. Thus, in response, administrators are forced (!) to limit password length just so their authentication servers can't be attacked.
This design flaw (imo) could be pretty easily solved by hashing the user provided password once first, and then using that hash to concatenate each round. This changes the complexity to `number of rounds + password length` (note the "+" !). As a secondary benefit, the multi-round hashing step actually becomes constant time because all inputs are the same length! In fact, I wonder if the current design makes it possible to infer a user's password length from timing attacks on the authentication servers while a user is logging in.
I've not got any satisfactory cryptographic reason why we still use this design for password hashing besides platitudes like "don't roll your own crypto", appeal to authority "It's been designed by the experts", or stupid reasons like "but network bandwidth!" -- as if an http authentication request that's 4kb is gonna slow down the network vs a 3kb request (and this is a worst case of a relatively huge 1kb password!). Availability is an important part of security, and sacrificing both availability (vulnerability to DOS) and security (admins forced to limit password length) seems like a doubly whammy argument against the current design.