Swap isn't some artifact from the days of 640k, used only because memory is expensive. Shit is always stored on disk; swap just allows that shit to be unused pages of active programs rather than actively used pages of files on disk.
Without swap, you force the kernel to prioritize cold code paths of rarely used daemons over, say, your web browser's cache. That's just dumb.