Much Modern Innovation Isn't
broadstuff.com
broadstuff.com
They gave us the transistor, Unix, and the C programming language. They hired people like Claude Shannon and Dennis Richie.
Then they broke up AT&T and now "innovation" at AT&T is peddling an inferior DSL service and trying to figure out how to get you to pay more for your cell phone bill.
YCombinator-style startups might make something like AirBNB, but they won't make something as transformative as the transistor.
As for lean and agile, these are both profoundly conservative forces, in fact, that's really their strength. At their best, they improve the odds of project success by squeezing out risk. However, they also encourage people to keep their heads down and not ask the kind of fundamental questions that can lead to 10x gains rather than 10% gains.
Lean and Agile act as a "hill-climbing" algorithm. You start at point x and take an action in some direction. If you improve, continue. If you don't, change directions. Keep doing this until you can't improve any further and you've hit a local maxima.
The important concept here is that you hit a LOCAL maxima. There could be other vastly superior outcomes, but whether or not you reach these outcomes depends on your starting point.
I think Lean and Agile are fantastic for making improvements to a product. However, in order to change the game, you need to take risks by jumping to a new starting point and beginning again.
I think Google, as the article points out, espouses both philosophies. They take massive lateral jumps (Driverless Cars, Project Glass, etc.), but once they make that jump, their development is Lean and Agile.
However I think it's a good point so I'll bite.
There are many reasons for Agile and Lean and there are also differing interpretations for what it means. However, generally it does mean continuous improvement as opposed to blue skies type innovation. Usually this is a good idea since it removes product development risks. A lot of people aren't very good at getting things done, aren't that good at coming up with creative ideas, and given too much lee-way will hide inside the red tape you've created for them; for most companies it's more important to get something out than it is to innovate. Similarly this is the reason for closed-allocation policies, etc.
Of course, if you are in the top 20% then you possibly could be ultra innovative, creative and productive in a less restrictive environment...
Not that agile or lean are about process of course, but the interpretation by most companies is very different from the original manifestos.
Speaking more loosely I'll take a stab. Innovation is a shift in the way things are done that either permits more value to be created with less resources, increases human freedom of choice, or that opens an entirely new avenue of human endeavor.
Innovations can be small or large, but when I think of innovation I usually exclude routine, linear improvements to existing things. For example, I don't consider a car that gets 32mpg instead of 30mpg an "innovation" but a "refinement of an existing innovation." But fuel injection was an innovation, and boosting fuel economy was one result.
One of the characteristics of innovation, I think, is the "leap." An innovation isn't just a purely logical, linear next step derived in a formulaic way from previous steps. There is some new information there, and the fact that it's very hard to say how this new information came into being seems one of the circumstantial characteristics of an innovation.
That is not to say it emerges out of the blue. It does come from study and focus. But it is not plodding and purely logical. There is some "eureka" that's hard to back-trace. I've had innovations come in dreams, or out of the blue in the middle of the day while doing something else.
All that being said, I think it's probably true that overly rigid structures inhibit it. Overwork can also inhibit it, due to its serendipitous and apparently "right brain" nature. Agile and Lean are all about increasing efficiency, and they often work that way, but they also impose a lot of rigidity in certain places.
A lot of rambling and philosophizing I know... trying to say something about innovation does that.
All creativity is about giving space and giving trust to people to express their creativity and create something amazing.
If by innovation you mean the invention of totally new and interesting things, no. There are plenty of new and innovative ideas/products that come from an agile/lean process.
What people are glossing over is the fact that building things is hard. Inventing things is hard. Changing the world is hard. It takes time, effort, the right people, the right timing, the right placement. Agile or waterfall don't matter. People do.
Smart, driven, clever, brilliant people are never going to be the majority. The majority of people are not above average. Expecting a large portion of people to become continuously more awesome every generation is insane.
Not everyone is going to be a winner. Most people aren't exceptional.
No amount of process of any sort can change that.
There is no magic bullet "methodology." Agile, Lean, etc. are sets of ideas and methods and some of them are very good, but they cannot be applied without thought. Think of them as management design patterns. You still have to look at these sets of design patterns and decide which ones work for your particular situation, or adapt them to your peculiar organizational needs. I've seen too many shops that apply Agile/Scrum/whatever slavishly and unthinkingly with poor (and often extremely annoying) results.
Probably more interesting to investors than developers.
But in my painful experience, large corporations seem to:
1. Apply all the micromanagement bits they like (daily standups, sprints) and ignore the rest of the methodology. This then creates a horrible Lovecraftian mutant methodology about which others have written, but IMO sucks.
2. Insist on a one-size-fits-all approach and broadly speaking, agile is fine for incremental development of software with solid and understood requirements, but it goes off the deep-end when there's any level of open-ended research required to get a satisfactory result. Ironically, this part is inextricably intertwined with the first point wherein fixed schedules and requirements are hybridized with daily standups and 2-week sprints.