You are confusing that recurrent networks are trained unrolled, i.e. with finite history, with them beeing evaluated with finite history, which they generally arent.

The key advantage a kalman filter has is in its long term properties being predictable, things like observability, controllability, and bibo stability.

The key downside of kalman filters is that their nice properties only apply to linear systems and linear observations.