There is no way a game like the first Portal could have been designed by a game designer, it started as a research project by a developer. Then there are games that do very innovative with the art blending it seamlessly into the design in unconventional ways, like Antechamber. I think there are plenty examples of games that buck the mold too hard for a conventional designer to have been able to make them.
My favorite example is Minecraft and its procedurally generated infinite world, much of the art is in algorithms that create the biomes. Cresting the top of a mountain to see the scenery on the other side for the first time is so simple to do and easy to enjoy in this game. The programmer had to be the designer and the artist to pull this off. I know Notch worked with others but this one facet had to be singular or iterative, not communicated from someone seperate after "designing". This was communicated in code and programmers and artist saw what existed and tried to build something and sharing. There is no good way for a conventional designer to convey the details of an algorithm to build forests and cave systems. It is not a fundamentally unsolvable problem, it is just presently unsolved in my experience.
Perhaps, for certain types of games. But for many types of games there's this subtle overlap between design and programming that defines how the game "feels" to play.
eg. How do digital/analog sampled inputs exactly map to character movement and camera control.
This is an area where having a small core group that understands the technology at play but also has a handle on the design side of how to implement for "fun" can be a huge difference maker.
From a high level design point of view, most action "platform" games have basically the same controls. But anyone who has played, say, Nintendo's Mario games versus most other platform games realizes there is some special sauce in the "feel" of the character movement that is equal parts design and programming/low-level-implementation that wouldn't get conveyed properly in a higher level outsource-ready design document.
If I want to be told stories I usually prefer books, films. (Though I liked Baldur's Gate for eg. which is an opposite of what I usually prefer)