Show HN: Run dedicated iOS Simulators in the browser | Hacker News Reader

Show HN: Run dedicated iOS Simulators in the browser

appetize.io

147 pointsweiyin67 comments

Open articleView on HN

Yo, dawg... http://i.imgur.com/Ib0WUeT.png

uh... oh boy. you're really pushing the boundaries of our testing... :)

Haha, I'm a little surprised since this was the first thing I did. Actually works pretty well.

not sure if you care about this one...http://i.imgur.com/OsGo10T.png

No worries, that's a mock location. Thanks!

I think that's the Apple store in San Francisco ;)

I got a little further... http://i.imgur.com/IVktzqU.png

Wow, I thought I was super clever for doing that, and everybody else thought of it first!

http://cl.ly/image/2L3y0c3G2q0j

Developers are all the same, how could I expect to be the only one thinking of this? Anyway, I made it down to the 4th level in appetize.io-ception before the sessions started timing out.

http://i.imgur.com/wCIXQUB.png

You should do this on the browser of an iPhone. :-)

This is really cool!

The "video" quality is pretty bad though. From what I can see, the app streams base64 encoded JPEGs via WebSockets and draws them in a Canvas. The overhead of sending complete images costs a lot of bandwidth.

You may want to look into jsmpeg for live streaming: https://github.com/phoboslab/jsmpeg

Thanks for the suggestion, that looks like a cool library. A future goal is to be able to provide higher resolution images to fully show off the high DPI displays.

Why not use WebRTC for streaming the video?

You don't even need to go that complicated. A WebM stream over HTTP is pretty easy and would work well for this.

we did some testing using compressed video, but ended up finding single frames to be more performant. also, the setup can handle much higher framerate and quality but we've kept things fairly lean for the demo :) can easily handle 15fps w/ high quality images. thanks again for the questions and comments.

It's worth noting that Chrome Remote Desktop uses VP8 (which is the video codec used in WebM):

http://googleblog.blogspot.com/2012/05/next-step-in-chrome-o...

If you're having performance issues with it, you just have the video codec tuned wrong.

definitely a possibility :)

we'll check that out, thanks for the heads up.

We've actually experimented with jsmpeg for this type of thing. The problem we ran into was that jsmpeg seemed to require every frame from ffmpeg. If the browser or network slowed, the feed would fall behind the simulation, leading to several seconds of latency when interacting.

We'd love to use it and appreciate your hard work though. Do you have any ideas on how that problem can be solved?

There are two sides to this problem. For jsmpeg (or even in the server), it would be possible to simply wait for the next intra frame (a frame that doesn't require previous frames to decode) on certain conditions and throw away all other frames in between. This will give the decoder and network time to catch up. By default ffmpeg emits one intra frame per second, iirc.

The more difficult part is probably detecting this lag. I'm not sure how this could be done. Maybe the node websocket server (ws) can detect congestion somehow.

Thanks. Maybe our ffmpeg settings weren't creating intra-frames somehow (or we happened to drop every single one) because the stream would never recover. I'll probably try it again at some point soon. We really want jsmpeg to work.

base64 encoded JPEGs

MJPEG is supported in all browsers I know of, and would cut down on bandwidth by 33% over base64.

(If you really want to use WebSockets, a more dense encoding than base64 could also help - look into base85, for example.)

MJPEG is good, but keep in mind that Chrome doesn't support playback of MJPEG streams as a main resource, it has to be in an <img> tag.

Looks like that functionality used to be there, was broken, fixed, and then broken again sometime later. The latest bug for it appears to be https://code.google.com/p/chromium/issues/detail?id=308999

For this application you'd need it to be in an <img> tag anyway since there's the other stuff on the page too, so that's a moot point.

Not only <img>. The MJPEG stream could theoretically be embedded in an <iframe>, thus being affected by this bug.

Running an iOS simulator in the browser is different from using a browser to connect to an iOS simulator on a server. That said, this is still cool!

My only question is how you're going to combat abuse. You're basically providing free servers in the demo. The economics of doing that means no amount of captcha will save you (it's been tried many times).

Even if you don't provide a demo and charge for simulating, you're buying into a whole new problem of metering while maintaining a sane price. Comments?

That said, congratulations on shipping working software!

hi tuscany,

all solid points, and thanks for your support.

we will definitely need to think through abuse, server costs, etc. it's a pretty lean setup but still gotta run on mac hardware :)

weiyin

Hi, thanks for checking us out. We're happy to answer any questions in the comments.

Some of you may be familiar with app.io's demo pages service which was discontinued in August. We loved using their service and we thought it'd be a blast building a replacement.

Technologies used: node, socket.io, mongodb, html5 canvas

I understand those are the web technologies, but how is this being run in the back? Tons of mac servers, with multiple IOS simulators on each? One running per user on the mac? Custom software to control and pass through to the socket?

Care to shed any light as to an overview on how the backend works? I've always been curious since I first saw App.io

> Tons of mac servers, with multiple IOS simulators on each?

Maybe a modified iOS Simulator binary that can run hundreds of instances on a single powerful server, would be my best bet.

Maybe, but then launching multiples and capturing the screen of them to send off through the sockets seems like there must be something in between. I wonder if its able to see where the session windows are on the screen and screenshotting them to send through, or if they somehow hooked directly in to pull the screen?

Hey B-n-c, You are pretty much spot on with everything you mentioned.

Apparently app.io had some setup w/ VMs running on lightweight hypervisors (https://macstadium.com/casestudies), but we found we didn't have to have all that overhead to run multiple simulators per machine.

And you're exactly right about the small piece of custom code to pass controls and send frames. We're big fans of socket.io - pretty powerful stuff!

Happy to answer follow-ups if we missed something.

Thanks, weiyin

The one piece that puzzles me is how the frames are being captured from the iOS simulator. Are you launching multiples per user account, where each one is a different session at a different portion of the screen, and essentially screenshotting each? Or did you modify the simulator somehow and are hooking in to grab the screen, so that way its irrelevant of their position on screen?

Thanks!

We wrote some custom OS X code to capture frames from individual processes. That way the simulators don't need to be positioned in any way or even visible. Cheers.

The other piece of magic you are doing is sending in the keystrokes / mouse movements. If you're running multiple iOS simulators on a Mac, how are the key presses / mouse movements not interfering with other users?

(don't see a link to reply under your last comment)

yea, so the incoming keystrokes and mouse movements are managed through some custom OS X code as well.

the specific method we're using is CGEventPostToPSN, more information here: https://developer.apple.com/library/mac/documentation/Carbon...

(No reply button under your other comment either)

Thats a pretty sweet API, never seen that before, thanks! I'm going to play around with it a bit to see what it can do!

As previously the Head of BD at App.io, I applaud your efforts! Well done! Quite a few ways you can monetise, good luck! :)

Likewise!! Well done guys. Great to see you've grabbed the torch. Good luck and I look forward to following your progress.

Thanks, it means a lot to us coming from you guys!

Can't wait until you figure out h.264 and the MediaSource API ;) You guys are moving at an incredible pace! Very impressed.

ps. I love the new /embed endpoint and iPad support. Keep it coming!

Well, this is cool, but Safari allows for loading basically any file located on the filesystem. For instance, trying file:///etc/passwd results in file found but can not be opened by Safari.

That could be an attack vector.

Could you explain where you think the issue here is? Are you saying, as a user trying to upload, you could possibly (presumably accidentally) upload your passwd, or as a user of their site, you could somehow see their passwd file?

If it's the former, I think that's a 'vulnerability' with any site that has a file chooser.

If it's the latter, could you elaborate where and how you would do so? Perhaps you're saying you could load it through the simulator that's running on the actual computer/vm somewhere, but those simulator environment are sandboxes (afaik) so that you could only access files on within the simulator's context.

Simulator instances aren't sandboxed, so you can transverse the master system's file system with a malicious iOS app. This is indeed an attack vector.

Huh, indeed they are not. Was able to pull a file off my desktop through Safari. Was not aware of that.

Agreed. This is an issue not just for Safari, but someone can upload a specially crafted app that browses the file system.

In this iteration we are using the OS permission system to isolate users. We are looking into more effective sandbox methods. Security is an ongoing process, and we always take an adversarial point of view when evaluating our system.

Just having the simulator isn't enough for us Xamarin developers. One big issue with Xamarin is that you need a mac (with xcode installed) on the network in order to do builds. If you could solve that issue along with the having the simulator online then you have my money.

Not ideal but you can use an iOS CI/CD / build automation service like Bitrise (http://www.bitrise.io/) (co-founder / CTO here) to do the builds for you. If the simulator service has an API you can even automate the connection with Bitrise.

There are a few online code compilers too like this Swift one (http://swiftstub.com/) to work with.

Certainly not ideal but probably not impossible.

P.S. at this time you're probably better with an OS X VPS.

More information about what the site does would be helpful. Also, I get this error on Firefox/Windows "no video with supported format and mime type found"

Edit: also click on the demo and saw the simulator but couldn't get it to do anything. What is supposed to happen in the simulator? Should I be able to click on something? Any instructions/information would be helpful. Right now you're assuming everyone knows what the site is about and what the simulator is supposed to do.

I had the same issue, using Firefox 33 on Windows 8.1. It looks like switching to Chrome fixed the demo. Would be nice to have some sort of notice that certain browsers aren't supported (if that's the case).

Also, fonts are rendering poorly in Firefox and Chrome [0]. Bumping the font-weight up to at least 400 makes the text much more legible.

[0]: http://imgur.com/2wVXvUr

Hi Michael.

Thanks for taking a look! And sorry our site is having some problems on Firefox. Maybe use Chrome for now until we fix it up.

This service we put together would allow anybody to run iOS simulators in the browser for a whole variety of reasons. For example, a developer can easily send builds to clients/colleagues or even embed the demo in a website or blog post.

Cheers, Weiyin

I'd be willing to pay money for this, seriously. I've followed you on Twitter - let me know if anything happens.

We were using and enjoying app.io[0] until they decided to pivot and stop serving us.

[0] they didn't support pinch gestures, which made using a map impossible

You may want to watch out for people using this a proxy to do bad stuff with your servers. I noticed that all traffic is coming from Georgia, not my computer.

There's market here for training corporate users over the internet to use line of business iPad apps.

I'd pay for it, if it actually made training easier...

Nice work! I might pay for a version of this to get our graphic designer off using Xcode (my app uses an art/config format which can be pulled from a web server). Especially if it could help them quickly test results on multiple device formats and iOS versions. It'd be handy to have a slower full-resolution option too!

Definitely on our to-do list. Stay tuned!

Could this be used without the frame to emulate iOS apps on the web?

I.e, with AutoLayout, can you not adjust the frame of the app to be that of Android, and therefore have a semi-legit looking webapp for Android without developing a responsive website?

I'm not sure what problem this solves but it's technically cool.

That was my initial reaction as an iOS developer would have access to Simulators via Xcode.

That said, I could see this as a tool for sharing to a client for demo purposes.

You can let users play around with your paid app to see if it works as expected. Certainly cleaner than messing with In-App-Purchases or a separate "Lite" version of your app.

I've heard in one episode of the Core Intuition podcast that Panic has actually spent some time polishing a simulator build of Transmit(?) for iOS, for use with app.io (a service which did the same thing as the OP until they pivoted). This is something I'd also love to do if this business promises to survive longer than app.io did.

This is especially interesting if you are an iOS freelancer who wants to show off their work, without clients having to create accounts etc. :)

It was a demo of Panic's DietCoda! It was running for a while but I think it's been taken down now (since App.io's pivot maybe).

It reminds me of Pieceable: http://www.pieceable.com/

Nice. I would consider turning off motion in the settings because it would look less quirky in the videos

Make it work with Xamarin (Windows).

Needs landscape! Overall very awesome service.

Agreed! We are also working on supporting additional device types (iPad, iPad mini, iPhone 6(+)).

Very cool, what lead you to develop this?

hang on. im stuck here.

http://imgur.com/F1gLIzb

Reply on news.ycombinator.com