We're doing something similar at our company. We're fine tuning small LLMs and using them as isolated components of a larger deterministic system. When we tried using LLMs for larger components of the system we ended up writing a lot of pseudo-code into the prompt. That was better off as just code. Now we can quickly find tune our business logic based on client preferences, without having to dig into the prompt. This also means we build trust in our prompts over time, because they change less often.