Security - Make sure this doesn't lead to a back door into people's devices. Or make sure someone doesn't utilize this to make your product undesirable.
User opt in - Give users the ability to opt in or not. Tracking information might be useful, but some people do not like it.
Rollback is an important feature. User control of this feature with notification is a bonus. I.E. notify the user an error occurred with the option to rollback to a release that was stable for them. Currently we provide a link to all past versions which they can download and install over the current version if there is an issue. Not the best solution, but it has worked for us.
Internet access for applications that do not need it. This is tricky, so providing another service or plugin might be the way to go.
Automation - Manually keeping track of problems and resetting or rollback features can be time consuming. An automated approach with user opt in would be great as well. Automated analytic info on top of it would be even better.
TwistedApps