I just checked my contract again. If the patent is outside the scope of my work activities, and I don't write up the application while I am at work, I don't see how they could claim it.
But then Americans have these funny "all your intellectual property are belong to us" clauses in work contracts....