Frustratring lack of conclusiveness but alas, such is science.

Hard to imagine how a reduction in neural cell/synapse volume, albeit acute, wouldn't have any lasting effects.

  "The effect of caffeine on the brain was noted as particularly relevant in the right medial temporal lobe. This area of the brain includes the hippocampus and is responsible for processes such as memory formation and spatial cognition."

If you become temporarily worse at memory formation, that stands to reason as being ipso facto bad for overall memory accumulation.