Yep! yeah man playing jazz is I think the ultimate challenge for a musician. I made the mistake of trying to lean on fake books for a long time, I think because I didn't believe that it was realistic to memorize enough of the tunes for it to make a difference. Eventually I wound up with a teacher who insisted I do exactly that. I think it should have been obvious to me, but as I'm sure you know, once you have memorized maybe a hundred tunes or so, you develop this mental model that the bulk of all standards fall into on the basis of form, changes, feel etc. Then, even when its an unusual tune, maybe you have to struggle through the first A and B, some kind soul might take pity on you and shout out the changes, and that little bit of extra information is enough combined with all that other stuff you know to get you through the tune.
I guess all that memorization is like the conditioning and training that an athlete might do to prepare for an event, and then at the event you're mostly responding conditions.