[article author here] As smat pointed out - absolutely, it's difficult! In the ideal case we'd have a proper 3D reconstruction of the scene; something like a light-field representation that permitted enough lateral movement in the camera extrinsics to adequately sample inside a synthetic aperture. This theoretical development is important, but it doesn't block getting to a point that is good enough for consumer use cases.

How it goes in practice - even for high-end VFX shops - is that the result looks and feels right. As an example - the depth maps used in high-end VFX (especially for stereo conversion) is sometimes hilariously and incredibly inaccurate (e.g. having depth on a foreground spaceship but also depth on a planet thousands of kilometers away).

Accuracy improvements in absolute modeling in the real world may not translate in a visible, meaningful way when put through a simulation that acts upon it; instead we focused (ha!) on improving the light modeling process from a generic 'textbook' example of kernel-based simulation to one that more closely matches how light moves through an actual physical camera.

And yeah, the colours are bit punchy but that's my fault - my photography aesthetic unfortunately is tuned towards ludicrous colour, and I probably need an intervention for someone to take my cursor away from the vibrance slider :P

It is actually impossible as a lens with large aperture gathers rays from behind close objects. So it is a problem with incomplete information.