AWS Device Farm adds support for iOS | Hacker News Reader

AWS Device Farm adds support for iOS

aws.amazon.com

66 pointsimpostervt19 comments

We found it hard to find reliable testers for iOS without building your own team or doing it all on your own devices. Maybe Amazon will come through, but we'll see.

I'd love to see a service where one super tech just tries to break your application on every supported device for a couple hours. A room full of toddlers might also work.

I think it would be a truly viable startup idea to connect app developers with "naive" people - soccer moms who simply haven't got the time to learn tech properly, 70-year-olds who've never used a smartphone, 3 year olds, etc.

Being able to get your website or app (or even good ol' desktop program!) tested by people who have <specific metric> exposure to <granular list of options> wrt. technology would be invaluable, I think.

I consider it sad that this either doesn't exist, or isn't yet a standard part of the common consciousness/culture.

Applause.com does this kind of "in the wild testing" with testers of all levels of expertise. Definitely worth checking out.

We created an Android AND iOS farm for automated game testing. You don't need to root iOS to get it done - it's just a PITA to do it right. You end up needing an attached Mac running XCode to send commands & you can't use the internet connection through the wire so we set up a local secure wifi network.

The only devices we had to "root" (aka put in developer mode) were some android devices that wouldn't allow us to change the HOSTS file otherwise ;-)

I'm glad that AWS is now offering this type of service, I can't wait to try it out.

This is admittedly definitely the kind of thing I'd love to see a "making of" about.

Actual real devices?! I would really like to get a bit of a behind-the-scenes on how they managed that. :P

Did they work with apple to create a custom version of iOS to integrate with amazon's service, or did amazon jailbreak the devices and installed their own hooks?

From their FAQ: "AWS Device Farm tests are run on real, non-rooted devices. The devices are a mixture of OEM and carrier-branded devices."

It's probably not rooted, but it probably also violates Apple's developer license agreement (which are needlessly restrictive) - sections 2 and 3 in this doc: https://developer.apple.com/programs/terms/ios/standard/ios_...

More importantly, what do they do to keep users from jailbreaking the device via software? Especially with Android, there are a lot of software root flaws and the privileged surface area is a lot bigger than the hypervisor AWS uses for EC2.

The summary makes it sound like they only wipe your app from the device, not reinstall the OS between users.

I believe it doesn't matter for 95%+ of users.

EC2 is used to run production services with sensitive data.

Most Android/iOS testing is done with made up data, without privileged access to production. So even if it get compromised, it probably not much different from installing an app on jailbreaked device be end user.

It says first 250 device minutes free. How much does it cost after that? There doesn't seem to be a direct link to the pricing page.

"Pricing is based on device minutes, which are determined by the duration of tests on each selected device. AWS Device Farm comes with a free trial of 250 device minutes. After that, customers are charged $0.17 per device minute. As your testing needs grow, you can opt for an unmetered testing plan, which allows unlimited testing for a flat monthly fee of $250 per device."

From the FAQ[1] which albeit was a bit buried.

edit: It is also at the bottom of the landing page now.

1: https://aws.amazon.com/device-farm/faq/

thanks!

> a flat monthly fee of $250 per device.

That's a bit steep? You could recoup any iOS device in <= 3 months.

I believe Google already provides this service to its developers for free.

Wait, Google doesn't support iOS devices (whereas, this announcement is specifically about iOS), right?