Also: if you're going to post a question like this, I'd suggest not doing it with a Hacker News handle that looks like a real name. Consider if your employer read this post before you told them you were leaving, or worse yet if you decided to stay.
One thing I learned over the years, there is no large company that truly depends on an employee. The machine will keep on ticking, even if it has a little bit of difficulty for a short (relatively) period of time.
To the OP: If you can handle your existing financial responsibilities with the change, then by all means go for it.
I wish I had followed it as much as I believe, but... if you do what you enjoy/love then enough money will follow.
Like you, I got sucked in to a job with the salary and then after getting married the spouse wasn't comfortable with taking the chance on leaving.
EDIT: As far as your friends go.. They are not the one's living your life. (yet another one of my mistakes).