The basic argument is that those who want restrictions to content have been attempting to chip away at what is essentially an open platform. While the W3C's argument is that it is an "extension" to the HTMLMediaElement, and while the draft goes to great pains to state that it is not a DRM system, in actual fact it goes a great way towards allowing for a DRM system.
If you review the diagram in the draft [1], you will see that it covers playback of certain content. Currently, it is restricted to media files. But you can see that once this is adopted, it would then lower the barriers for media companies and other interested actors to push for encryption mechanisms for individual elements.
Even worse, note that the Content Decryption Module (CDM) is a "part of or add-on to the user agent that provides functionality for one or more Key Systems". In other words, these will probably need to be implemented as binary blobs which are add-ons for browsers. They may need to use specific technologies that are part of particular operating systems.
What does this leave us all with? Well, it leaves us with the promogulation of a bad idea (restrictive DRM) implemented using a variety of browser specific extensions that may need to utilise the specific technologies of a particular operating system.
As an example, you could be forced to use an add-on for Internet Explorer that can only be used on Windows 8.
So in other words, the plan is a bad one for technical reasons, and for political reasons it lowers the barriers to campaigning for more restrictive measures leaking nto the rest of the standard.
1. https://dvcs.w3.org/hg/html-media/raw-file/tip/encrypted-med...