Consistency / familiarity is a very important part of developing UIs/UXs that are intuitive.

What you have with material design is a very well documented approach to promoting this. The wrong direction? As a developer I lack the expertise/talent to have an opinion one way or the other on the direction presented by material design. But if the designer community is unhappy with the approach, then they should probably get together to create competing principles that can be adopted by the rest of the dev/design community.

You are still free to design you applications however you see fit, but you risk missing out on users quickly feeling comfortable in your applications.