Of course, you could just go download the APK from somewhere else.

I doubt many people (myself included) would have -any- issues with the App Store's restrictions if, like the Android Market, it wasn't the exclusive and only way for distributing applications, and you could distribute the binary like you can distribute an EXE or DMG.