I can't speak for OP, but something that I think helps is if you think about the generation process as a jumping off point that one can control the placement of, but not really much that is generated afterwards.
Adding a scheme like this reduces the area of potential off-roading that the LLM can do to a much smaller zone. Additionally, it breaks up the chain of dependencies between the two example outputs, because now we do not need to depend upon past inputs to correctly output this scheme.
Since the information for JSON semantic structure is no longer required to be driven by the LLM (it still has to understand it to still be able to generate things with a modicum of sense, IIRC), we can look at our dependency graph for outputs. _This changes because now the fields really and truly are independent, (if they are truly informationally independent) _.
So now some kind of conjoined information requirement of ( autoregressive output ) <- (( field A ) <- ( field B )) becomes ( autoregressive output ) <- (( field A ) && ( field B )) which then can be factored out into separate calls instead of sequentially, which yields us a batched call of (( autoregressive output A ) <- ( field A ) && ( autoregressive output B ) <- ( field B )).
From there it is just implementation. I likely would not have thought about the OP's way of handling things for a good while, though maybe I would have stumbled into it had I enough reason to think about structured/templated kinds of generation, which I do believe that I do now! <3 :) It really breaks a lot of assumptions that are easy to quietly make and I had not thought appropriately about the consequences of reframing things in this way, to be honest.
As for "how" to think about this, if I were to give my take, it would be always just turning whatever problem in front of you is into a puzzle where you simplify it further each time. Optimizing for less computation, time, code, or even just what all of those are a kind of proxy for: less information to sufficiently solve a problem. We can see that this problem is reduced in complexity appropriately because we remove a redundancy that does not need to be there at all.
One way to look at this is in the relationships between parts of an idea. If you're able to understand, even vaguely, the concepts behind some other concept and how they interact, and maybe even have a 'standard toolkit' of relating to them, you can start finding/transferring/applying other skills to these parts of a concept. I don't think there's a guaranteed-efficient way to maybe reduce a concept down to its parts, or down to a more efficient representation without already, well, knowing that representation. It's an NP-hard problem to me personally, and is the reason why research and other academic pursuits can take a while. It is a good skill to learn I suppose and I certainly enjoy trying to use it, personally.
To tie this back to your question about language models -- yes, some things have to do with the language model, but oftentimes it's actually just the raw mathematical components underlying a model. If you look for that, and (please please please please please!!!!) then you don't necessarily _have_ to concern yourself with the implementation details (beyond runtime limits, etc), as long as the math still applies you should be able to reason really quite well about what else is happening/could happen with a model type like these are.
In particular, LLMs being an autoregressive model where each output depends upon its inputs lets us set up a dependency graph. Then based upon some prior assumptions, we can maybe make some substitutions/changes that allow us to fragment the dependency graph and move it around as we wish. This is not just applicable to LLMs, however, dependency graphs are useful in a wide number of areas.
So one other thing that we're not talking about here is that we're optimizing for an objective we want (clean JSON) by explicitly...well, injecting that objective instead of living on just hopes and dreams, y'aknow. This is a pretty straightforward way of solving the problem by putting the answer in the question, though poor input content still can be a problem.
Stated a different way, we're collapsing the entropy of what the network can introduce (which should be JSON, but remember [!!!!!!!], neural networks are noisy estimators, and JSON errors are mathematically guaranteed (even if rare), which means any pipeline depending upon output like code can and will fail, and is brittle to all sorts of other kinds of complicated parsing errors. This is because to catch/detect/enumerate/correct these errors, we need to have all of the information needed to implement a JSON structure itself. So basically we'd be using the same exact information, just enforcing it in a horrendously inefficient manner, which is how people have been doing it until the present, which is okay as we humans are certainly not NP-optimal machines IMO. In any case, we're still in the parentheses, and the point was that any kind of variance can be a problem here beyond some extremely tiny limit, and that's not what LLMs are made to do. So at some point it's guaranteed to break, and high volumes -- it's basically guaranteed to break in a way that's either unusable or requires so much effort to fix that you might as well have embedded a JSON prior into your network generation process because it would have required the same amount of information as external validation would, albeit with less effort (!!!!)), which is perfectly fine in our case if we're exclusively generating JSON as it gives us what we want. Most methods like this thankfully should have a low level of invasiveness to the model as well, freeing us up to use either the same or a similar model for multiple tasks.
This can create a bit of an ideological illusion as we technically are destroying information by collapsing the distributions of sentences/strings of tokens/etc that we are generating, and maybe can lend to a "oh, we can add whatever functionality we want!" kind of belief about this kind of modeling. It's important what we're adding and taking away. Also important is part of how/why/what is so powerful about training these models on next token prediction on large text corpora. We can trim them down to some smaller subproblem much much more easily than we can expand them to cover a larger subset. Which is pretty darn cool!
I know this sorta flew around a lot of places and touched on a lot of things, probably not as cogently as I'd want to if I had more time to review and revise it. Hope it was/is helpful for you and feel free to let me know if you have any questions. It's a very cool topic on the whole to me, tbh, and there's a number of interesting conversations that can branch off from this one. Honestly this whole general area is where I see the real value in LLM development in research. It's practical and it's helpful! :D :) <3 :)
Source for experience is a number of years of experience across a wide variety of ML models, though I'm sure I made an embarassing blunder or two in this post. ;P