Agreed, but it's that way for a purpose no? Maybe not a whole 20%, but enough of a buffer so that you won't be underwater on a mortgage in a down housing market.
Hopefully, you can save at a rate higher than the rate at which the down payment increases due to house prices generally going up.