As someone who's worked in security at a couple of quickly-growing tech companies and is now back to a non-security engineering role, I would suggest being introspective about what you find fulfilling about software development and think about whether working in security will offer enough fulfillment to go to work every day.
At its best, security is about seeing what systems _can_ do rather than what they're intended to do. On the offensive side, this means doing research on where systems are weak or simulating attacker behavior. On the defensive side, this means deeply understanding systems in order to control their behavior and mitigate risk, building automation to avoid human fallibility, and ensuring you have the necessary visibility into your systems to detect when your defenses fail or your assumptions are violated. This type of security work can be very fulfilling, especially when finding ways to improve the security of your systems in a more usable way (or at least not-less-usable then before).
It's worth mentioning that at larger organizations, the security team typically can't accomplish their mission alone, and must collaborate with other engineering and non-engineering teams to learn what problems exist, get to a shared understanding of how important each problem is to address, and then go on to solve them. This means that at larger organizations, security can't really be successful or a fulfilling place to work without leadership that sees the value of security and is willing to prioritize and support it. Unlike software engineering teams which are likely increasing revenue, reducing cost, or doing something else more visible and valuable to company leaders, it's easy to view security as a cost to be minimized because it's value is harder to understand or compare.
At its worst, security is more a risk management process than an engineering one. There may well be legitimate business reasons for this, but it makes security much less fun to work on. There are any number of specific failure modes here, but some that come to mind: security becomes focused on scanning, testing, documenting and prioritizing problems, doing the bare minimum to address the worst problems to CYA; security is tasked with dragging an unwilling organization along a path of risk reduction that makes everyone unhappy through the process; or security is endlessly reactive to incidents and vulnerabilities that arise, and has very little time or energy to make things better.
If you can do security work at its best, it's a pretty awesome field to work in. On the other hand, from the "at its worst" perspective, software engineering looks a lot more fun. As you wade into the security field you should be aware of the organizations and types of work you pursue to ensure you'll be able to stay motivated and keep growing regardless of whether you ultimately choose security or not.