I don't think that's what Plato is getting at. In many ways, Plato's theory of forms discounts the value of direct experience.
Firstly, look at memorizing as a workout for the brain. If you don't need to do as much of it, you do less of it, and you don't develop that ability as much. Socrates valued a fit mind and body and likely would have considered a good memory as an important part of mental fitness.
Secondly, consider the context. Plato was a student of Socrates, who taught not by lecturing on what he thought to be true, but by posing questions to his students and discussing various possible answers. Wisdom was obtained not by passively absorbing, but by dialogue with a teacher. Plato's works take the form of a passive listener, observing such dialogs. Later in the dialogue, Socrates gives this second objection to writing in more explicit form: "Writing is unfortunately like painting; for the creations of the painter have the attitude of life, and yet if you ask them a question they preserve a solemn silence."
Plato does seem to try to transcend the limitations of written form here. When Socrates relates the tale of Thamus and Theuth, Phaedrus pushes back saying Socrates can easily make up tales from foreign lands. Socrates retorts that it was enough for the men of old that they heard the truth, the source being irrelevant. Phaedrus too easily crumbles and concedes that King Thamus was right about writing. One is left with the impression that neither person has made good arguments against each other, leaving one to consider the issue further.