He has however really turned this into a personal brand which many suspect he is using to advertise his startups. Many people feel he is sort of disingenuous about the actual progress being made in computer vision and NLP, even if there is many ways to go.
To answer your question: You can read a single piece of his and maybe get some value from that, but he keeps rehashing the same points to sh*t over different new work, and there is not much to be learned from that after a while.
Disclaimer: I did not read the above article in full, I am just familiar with Gary Marcus + work in ML industry.
Edit: Keep expanding this but it got me thinking. Earlier in my career, I was also eager for the 'gotchas', for being able to point out why something is crap, exaggerated, cannot do what it promises, and so forth. As I outgrew this impulse to focus more on what is good about some work, I began spotting it on new students a lot. They would come into seminars and smugly crap all over recent papers.
I have since come to the conclusion that it is useful to have realistic internal assessment of progress in your field, but that sharing your harshest criticisms is not necessarily a good way to look smart or make progress.
It often turns out that while feeling clever about realizing limitations of some work, the field just moves forward. The people you just criticized leapfrogged while you were busy thinking about how to one-up them.
Someone definitely needs to stick machine learning's nose into its own dirt, for its own good first of all, but it must be done in a very serious, stick-up-its-arse manner, focusing only on the practical aspects and leaving any emotion aside, and all in a calm and erudite voice, otherwise it won't have any effect.
Of course, now that I say this, there has been one such critique and it was ignored roundly, probably because Noam Chomsky Doesn't Tweet.
This is what I mean:
http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html
Sorry, I can't find the edited interview (and the transcript doesn't quite get the "calm and erudite" voice across, what with all the uhs it transcribes). And I'm half-wrong about the reaction because there was a reaction, from a certain Peter Norvig, oh yes sirree there was:
http://norvig.com/chomsky.html
But I'm still with Chomsky on this.