It depends a lot of what you do with your phone tbh, I'm on a pixel 3a and still see no reasons to update, the fairphone 4 is better spec'd than that
I wouldn't run it with anything personal on it.
I very much doubt anyone is making kernel or vendor updates to upstream neglected components.