Is there any option to run a test suite before deploying? Otherwise this is about as dangerous as fixing code on a production server.
Some blog coverage here: http://blog.cloudbees.com/2014/03/jenkins-man-behind-curtain...
(Note: I work on Cloud at Google, although not on this part of it. Thus I'm going to be a little conservative and only discuss things I know we've announced)
edit: Oooooh, or just ask the TL: https://news.ycombinator.com/item?id=7617385