Seniority really isn't worth much on a software team because no matter how high you climb, there's always someone above you who can override your decisions.

Once you have stayed at a company for more than 3 years, leaving for greener pastures is never counted against you. If you think it's good for your career, do it.