I've been trying to dive into this myself and there are a paucity of good resources on the subject. One that I found really good though was Longuski's Optimal Control with Aerospace Application. It has a good balance of math and application, even some MATLAB code at the end to implement some of it.
One note I'll make is that usually there's 2 separate optimization programs for a rocket. The first stage runs on "open-loop" guidance, i.e. a trajectory is created beforehand (and by trajectory I mean something like a pitch-velocity lookup table - when you are at this velocity, pitch by this amount) and the rocket flies this trajectory without optimizing it in real time.
After stage separation the upper stage will run closed loop guidance (typically something like Powered Explicit Guidance or PEG) that actually takes into account orbital parameters and fixes error in inclination, RAAN, apogree, perigee, etc. that may have been introduced by the first stage.
I've worked at one aerospace company where I know they follow this pattern, and I believe others do as well though can't confirm from personal experience.
Longuski's book mostly deals with the latter part of things. I haven't found a good reference for how to create an optimized first stage trajectory.