I would do it now; it's not going to get easier later.

I personally have had good luck upgrading small apps very easily from 3.2->4.0. The main thing I've run into with larger apps is going from attr_accessible->strong_params, which I realize I could do simply by adding the gem, but I'd rather go whole hog on it.

If you're coming from 3.1 or 3.2, I say go for it. I have brought a couple applications from 3.2 -> 4.0rc and both had specs fully passing within 30 minutes of the switch. git checkout -b rails4, it won't hurt to try.