Developers are all the same, how could I expect to be the only one thinking of this? Anyway, I made it down to the 4th level in appetize.io-ception before the sessions started timing out.
The "video" quality is pretty bad though. From what I can see, the app streams base64 encoded JPEGs via WebSockets and draws them in a Canvas. The overhead of sending complete images costs a lot of bandwidth.
We've actually experimented with jsmpeg for this type of thing. The problem we ran into was that jsmpeg seemed to require every frame from ffmpeg. If the browser or network slowed, the feed would fall behind the simulation, leading to several seconds of latency when interacting.
We'd love to use it and appreciate your hard work though. Do you have any ideas on how that problem can be solved?
There are two sides to this problem. For jsmpeg (or even in the server), it would be possible to simply wait for the next intra frame (a frame that doesn't require previous frames to decode) on certain conditions and throw away all other frames in between. This will give the decoder and network time to catch up. By default ffmpeg emits one intra frame per second, iirc.
The more difficult part is probably detecting this lag. I'm not sure how this could be done. Maybe the node websocket server (ws) can detect congestion somehow.
Thanks. Maybe our ffmpeg settings weren't creating intra-frames somehow (or we happened to drop every single one) because the stream would never recover. I'll probably try it again at some point soon. We really want jsmpeg to work.
we did some testing using compressed video, but ended up finding single frames to be more performant. also, the setup can handle much higher framerate and quality but we've kept things fairly lean for the demo :) can easily handle 15fps w/ high quality images. thanks again for the questions and comments.
Thanks for the suggestion, that looks like a cool library. A future goal is to be able to provide higher resolution images to fully show off the high DPI displays.
Running an iOS simulator in the browser is different from using a browser to connect to an iOS simulator on a server. That said, this is still cool!
My only question is how you're going to combat abuse. You're basically providing free servers in the demo. The economics of doing that means no amount of captcha will save you (it's been tried many times).
Even if you don't provide a demo and charge for simulating, you're buying into a whole new problem of metering while maintaining a sane price. Comments?
That said, congratulations on shipping working software!
Hi, thanks for checking us out. We're happy to answer any questions in the comments.
Some of you may be familiar with app.io's demo pages service which was discontinued in August. We loved using their service and we thought it'd be a blast building a replacement.
I understand those are the web technologies, but how is this being run in the back? Tons of mac servers, with multiple IOS simulators on each? One running per user on the mac? Custom software to control and pass through to the socket?
Care to shed any light as to an overview on how the backend works? I've always been curious since I first saw App.io
Hey B-n-c,
You are pretty much spot on with everything you mentioned.
Apparently app.io had some setup w/ VMs running on lightweight hypervisors (https://macstadium.com/casestudies), but we found we didn't have to have all that overhead to run multiple simulators per machine.
And you're exactly right about the small piece of custom code to pass controls and send frames. We're big fans of socket.io - pretty powerful stuff!
Happy to answer follow-ups if we missed something.
The one piece that puzzles me is how the frames are being captured from the iOS simulator. Are you launching multiples per user account, where each one is a different session at a different portion of the screen, and essentially screenshotting each? Or did you modify the simulator somehow and are hooking in to grab the screen, so that way its irrelevant of their position on screen?
We wrote some custom OS X code to capture frames from individual processes. That way the simulators don't need to be positioned in any way or even visible. Cheers.
The other piece of magic you are doing is sending in the keystrokes / mouse movements. If you're running multiple iOS simulators on a Mac, how are the key presses / mouse movements not interfering with other users?
Maybe, but then launching multiples and capturing the screen of them to send off through the sockets seems like there must be something in between. I wonder if its able to see where the session windows are on the screen and screenshotting them to send through, or if they somehow hooked directly in to pull the screen?
Well, this is cool, but Safari allows for loading basically any file located on the filesystem. For instance, trying file:///etc/passwd results in file found but can not be opened by Safari.
Could you explain where you think the issue here is? Are you saying, as a user trying to upload, you could possibly (presumably accidentally) upload your passwd, or as a user of their site, you could somehow see their passwd file?
If it's the former, I think that's a 'vulnerability' with any site that has a file chooser.
If it's the latter, could you elaborate where and how you would do so? Perhaps you're saying you could load it through the simulator that's running on the actual computer/vm somewhere, but those simulator environment are sandboxes (afaik) so that you could only access files on within the simulator's context.
Agreed. This is an issue not just for Safari, but someone can upload a specially crafted app that browses the file system.
In this iteration we are using the OS permission system to isolate users. We are looking into more effective sandbox methods. Security is an ongoing process, and we always take an adversarial point of view when evaluating our system.
Just having the simulator isn't enough for us Xamarin developers. One big issue with Xamarin is that you need a mac (with xcode installed) on the network in order to do builds. If you could solve that issue along with the having the simulator online then you have my money.
Not ideal but you can use an iOS CI/CD / build automation service like Bitrise (http://www.bitrise.io/) (co-founder / CTO here) to do the builds for you. If the simulator service has an API you can even automate the connection with Bitrise.
There are a few online code compilers too like this Swift one (http://swiftstub.com/) to work with.
Certainly not ideal but probably not impossible.
P.S. at this time you're probably better with an OS X VPS.
More information about what the site does would be helpful. Also, I get this error on Firefox/Windows "no video with supported format and mime type found"
Edit: also click on the demo and saw the simulator but couldn't get it to do anything. What is supposed to happen in the simulator? Should I be able to click on something? Any instructions/information would be helpful. Right now you're assuming everyone knows what the site is about and what the simulator is supposed to do.
I had the same issue, using Firefox 33 on Windows 8.1. It looks like switching to Chrome fixed the demo. Would be nice to have some sort of notice that certain browsers aren't supported (if that's the case).
Also, fonts are rendering poorly in Firefox and Chrome [0]. Bumping the font-weight up to at least 400 makes the text much more legible.
Thanks for taking a look! And sorry our site is having some problems on Firefox. Maybe use Chrome for now until we fix it up.
This service we put together would allow anybody to run iOS simulators in the browser for a whole variety of reasons. For example, a developer can easily send builds to clients/colleagues or even embed the demo in a website or blog post.
You may want to watch out for people using this a proxy to do bad stuff with your servers. I noticed that all traffic is coming from Georgia, not my computer.
Nice work! I might pay for a version of this to get our graphic designer off using Xcode (my app uses an art/config format which can be pulled from a web server). Especially if it could help them quickly test results on multiple device formats and iOS versions. It'd be handy to have a slower full-resolution option too!
Could this be used without the frame to emulate iOS apps on the web?
I.e, with AutoLayout, can you not adjust the frame of the app to be that of Android, and therefore have a semi-legit looking webapp for Android without developing a responsive website?
You can let users play around with your paid app to see if it works as expected. Certainly cleaner than messing with In-App-Purchases or a separate "Lite" version of your app.
I've heard in one episode of the Core Intuition podcast that Panic has actually spent some time polishing a simulator build of Transmit(?) for iOS, for use with app.io (a service which did the same thing as the OP until they pivoted). This is something I'd also love to do if this business promises to survive longer than app.io did.
This is especially interesting if you are an iOS freelancer who wants to show off their work, without clients having to create accounts etc. :)
