Inside an Isotemp OCXO107-10 Oven Controlled Crystal Oscillator

It'll vary over time with the ambient temperature. When you set up a temperature control loop, you have a heater which creates a temperature gradient between it and the ambient temperature. This temperature gradient depends on the power that the heater is putting out, the thermal resistances of the box and any insulation around it. You then have a temperature sensor, which you will presumably put somewhere in the box, hopefully near the crystal, but it won't exactly be the crystal. Then, as the ambient temperature sensor varies, the power output of the heater will vary to try to keep the temperature that the sensor is seeing constant. But because that power also changes the thermal gradient, the thermal gradient in the system will also change, and so the temperature of the crystal is never completely insensitive to the ambient temperature, even if the temperature sensor reading doesn't change at all.

How big and/or meaningful this effect is depends on what the thermal resistances in the system are, and where the temperature sensor is relative to what you want to temperature control. Generally you want a very conductive box that's then insulated very well, since this means the temperature won't vary much across the box (all of the temperature gradient between the heater and ambient is 'taken up' by the insulation). But if you're talking sufficiently high precision this can be quite difficult to achieve.