The primary optimization that ASM.JS brings is its static type system. With that, and the obvious benefits to the code path, intensive activities can be run at 1/2 native speed, versus the 1/12th of so native speed available with dynamic typing.
Is 1/12th native speed "slow"? In most cases -- where the code is the duct-tape that glues together native code (which is the case on web servers, admin scripts, etc), no, it isn't slow at all. But from a basic optimization perspective, asm.js sure seems to indict dynamic typing if you really care about squeezing every cycle's maximal output.