Did you improve? Did you provide the results needed by the organization? Did you work better (not simply harder)?
Forgive me if this seems like an overly familiar set of questions, I'm not quite sure how else to frame them.
It's great to be loyal; it helps both parties make long term investments, engenders trust, and ensures stability.
But loyalty to a bad manager is no good. Similarly, loyalty to a report who is over their head and continues to be unable to improve at the speed needed is not good.
Source: I have both been over my head and had reports in over their head.