I devoted several years to this, and there are a bunch of reasons why that ends up being the case. The main issue is, think of how many atoms there are. Now realize the actual number of atoms is for all intents and purposes infinitely larger than whatever you thought of.
Ok, so we try to approximate. Right away you run into problems. What we call "light" is just a set of photons entering our eyes. Each photon has a wavelength. Our eyes do a compression algorithm in our retina -- our brains do not receive a "count" of the number of photons entering our eye. Instead, our retina cells themselves are responsible for coming up with an average of all the photons hitting it, and sending a signal to our brain: that's color.
Anyone who has any graphics experience whatsoever will know that it's an extremely common operation to calculate lighting by multiplying a light's color with an RGB texture. (The leaf's texture, say.) Now, tell me: What does it mean to "multiply" two colors together?
Nothing! It's fake! It has no basis in reality whatsoever. It literally is an astonishing approximation that happens to work reasonably well.
But our brains know the difference, inchoately, when you look at a real video vs a computer-generated video. You can tell that there's something almost indescribably "off" about the CG video. And I suspect it's because the final product consists of a series of approximations that work well in isolation but are not quite perfect when it comes to simulating reality.
Combine that with a lack of detail in CG video due to the fact that there are a gazillion atoms in real life and only a few million triangles in CG life, and you will always end up with experiences that are very distinctly computer generated.
Defeatist! Call me names! Say we can do better! Say that the latest advances in graphics and raw computing horsepower will solve these concerns. Maybe they will. But most importantly, I dare you to get so frustrated with our lack of ability to generate truly realistic computer video that you take on the problem yourself, and dive into the field with your whole heart. Surpass me. Surpass what everyone thinks is possible. Question the fundamental assumptions that the entire field is based on: that `N dot L x diffuse` is even slightly reasonable. Create your own "diffuse" textures that stores counts of photons bucketed by wavelength, measured from a real-life data source, instead of RGB colors. Try fifty things that everyone dismisses as unpromising, because whatever the final solution looks like, it's going to be unlike anything we're doing right now.
Cracking the problem of realistic video isn't going to happen with incremental improvements of our current techniques. Remember this rule of thumb: A video that's indistinguishable from real life will appear equally real to both humans and animals. A cat, for example, would think your video is real, if it's viewing your video on a monitor calibrated to cat eyes instead of human eyes.
It's more productive not to take the field too seriously, and to fall in love with the endless puzzles and challenges that graphics programming affords. Once you accept that the game is to generate something that looks cool, not that looks real, everything falls into place.