Just follow your heart, seriously. You only have one life so live it.

If you are really that critical I would give them a month notice which is more than enough time to find a replacement. That is the best you can do. If you give them plenty of notice, they will have no reason to think you are screwing them. You'd be screwing them if you just quit and left the next day.