I go back-and-forth in my career between being a manager and being an engineer. I find both sides rather fun for different reasons. There's a lot to unpack in your reply:
1. You're not acting in the role of an engineer. You don't know (intrinsically) what needs to be done. You just know what you want done. Engineers are probably doing shit behind your back and you'll never know except that your velocity seems slower than normal occasionally. You probably won't even have a slight clue unless you are an engineer yourself. Some hints might be when you prioritize a bug and your engineer tells you 'oh, that got fixed when we did X which seems totally unrelated to X, or just barely related'. POs understand this when the customer wants X but really needs Y. The same thing applies to you, engineers know you want X, but really, you need Y. When you pressure them for X, and you're not listening when they tell you you really need Y ... think about that for a bit. Have a conversation with the POs.
2. You should never, ever, ever, agree to anything without discussing with the team first. There should never be any surprises, because when you get surprised your response should be "let me discuss this with the team and get back to you before I give any commitment on that, but I'll get back to you before the end of the day." A key question on any kind of deadline is "is this a soft- or hard-deadline and why? How much wiggle room do we have?"
3. If you are getting surprises on any kind of annual basis or more, it's because you've become a "yes-man" and your team is paying the price. You're allowed to say "no" or "we're dealing with too much, can you help reduce the load" or "we can do it, but it will be two weeks later than you need it, here's what we came up with to get most of it done by X, and complete it by Y.".
4. Don't be a dick. Software Engineering is a 24 hour job. You dream about the problems you are trying to solve sometimes. It really sucks to come in to work, with a solution you've been thinking about all night and all morning, only to be told you can't do it because you've got a dick boss who agreed to some bullshit without even talking to you about it.
> Its the prisoners dilemma, and works when everyone works together but the minute someone stops trusting me and the team, it wrecks it for everyone.
It goes both ways, you have to work with the team as well. And regardless, nobody is a prisoner... hopefully.