> The only way to keep non-numbers out of your number input is to cancel keydown events selectively

There's a much simpler way: monitor the "input" event, and whenever fired, do this.value = this.value.replace(/\D/g,'');

The user will never see anything they type other than numbers.