The Pains of iPhone Ad Hoc Beta Testing

Interesting. This problem is tractable with better software, and it seems to be well within the capability of a single developer to solve. (Make a dedicated beta client which automates all the stupidity for the end-user, including sniffing GUIDs, downloading your distribution, and reporting errors to you, and a web service for the beta clients to connect to to grab apps, report errors, and cough ensure that your customers succeed in paying for your application.)

Unfortunately, while it is software that saves people pain (a plus) and helps make people money (a big plus), it is also sold to developers (ouch), most of whom are hobbyists (double ouch), would have to cost an awful lot in comparison to what they charge for software (triple ouch), and requires Apple to remain clueless or else your investment gets vaporized with a patch.

A couple tips to address some of his issues:

Issue: .app directory is browsable, don't use Vista to decompress.

Solution: Package your app as an .ipa. It is now no longer a directory, and your use can drag and drop this into iTunes. iTunes will handle the decompression. I usually send the following link to the users to help them through the install: http://www.innerfence.com/howto/install-iphone-application-a...

Issue: user needs to delete old version in order install new version. Can't test upgrades.

Solution: Increment your Bundle Version for each version.

Ad-hoc testers using Windows Vista have a couple of obstacles to overcome.

Not only the default unzip problem the post mentions (I've found it's just simpler to give the Vista testers an ipa file instead), but also removing the provisioning profile from iTunes usually involves going into the local filesystem and removing the .mobileprovision files manually: http://denis.papathanasiou.org/?p=139

"Over 10% of our testers gave us the wrong number as their UDID."

It's not that hard. There's a free app you can put on your device that will email your UDID to where ever you want, or you can just copy it over from iTunes/XCode. I haven't ever had a client give me the wrong UDID (they have given me the same UDID twice though, forgetting that I already have that device in the database). The hardest part usually is that it can sometimes take 5-6 tries of dragging the app into iTunes (on Windows at least) for it to work. But even that isn't too big of a deal, just quick and easy repetitive motion (once they get the profile installed).

I think the biggest pain in the ass is on the developer's side. Anytime you want to run the app on another device, you have to add another UDID to the device list on Apple's site, regenerate your provisioning profile, download it, delete the old adhoc profile from ~/Library/MobileDevices (good luck trying to find out which of the profiles in there is your old adhoc, the files are named with a random string of letters and numbers), install it to XCode, open your project, and change your adhoc build profile settings to the new profile.

Phew.

Agreed. I often have to give people my UDID several times ... and then sometimes it still doesn't work.