I think he's also giving thoughtful people hints about how they can mitigate the problem:
"..it will have become very hard for you to learn from anybody who doesn't have these clearances. Because you'll be thinking as you listen to them: 'What would this man be telling me if he knew what I know? Would he be giving me the same advice, or would it totally change his predictions and recommendations?' And that mental exercise is so torturous that after a while you give it up and just stop listening. I've seen this with my superiors, my colleagues....and with myself.
"You will deal with a person who doesn't have those clearances only from the point of view of what you want him to believe and what impression you want him to go away with, since you'll have to lie carefully to him about what you know. In effect, you will have to manipulate him. You'll give up trying to assess what he has to say. The danger is, you'll become something like a moron. You'll become incapable of learning from most people in the world, no matter how much experience they may have in their particular areas that may be much greater than yours."
And this advice is relevant beyond the realm of classified information. Most people are likely to find themselves in situations where they have what they regard as privileged information/perspective that in some way they can't share with others in discussion. Remembering that other people may still have valuable contributions to make in that discussion (and perhaps even a perspective that you don't have) can help ensure what you know doesn't keep you from learning what you don't.