Working with models like GPT-4 is frustrating from a traditional software engineering perspective because these systems are inherently unreliable and non-deterministic, which differs from most software tools that we use.

That doesn't mean they can't be incredibly useful - but it does mean you have to approach them in a bit of a different way, and design software around them that takes their unreliability into account.