This approach ends up blocking you from receiving updates due to do incompatibilities between your changes and upstream so it's not a great solution.
But you will have to do more by hand and yes, then there is also a chance that you mess up those patches and create new vulnerabilities. But if you have a big budget ... like if you are a big military and security really matters, than it is probably worth it.