I've been playing around with data extraction using OpenAI's structured outputs feature last week. Perhaps this is due to me still being fairly new to this world, but man, has it been an exercise in frustration. My general takeaway so far is that LLMs are fantastic for creative writing, and are a much harder beast to tame when you're looking for exact, reproducible answers. Even when those answers should be the most obvious ones. My engineering brain does not appreciate all the prompt engineering hacks, the hopeful JSON schema tweaking, the "let's try this other random thing just to see if it works" approach to making progress. The worst has been getting to an OK point with one of the fields, only to find out later that giving directions for another field blew away the earlier progress. I'm sure I must be missing something since there are products that are doing this, but so far I have little to show for hours spent other than growing knowledge of LLM quirks and subtle behaviors.