Are people here really suggesting we replace a mouse driven hover state on traditional computers with a touch driven tap and hold on touch screen devices? How does that make one lick of sense? What do you do while you're tapping and holding? Use another finger to navigate around your hover based controls? This is so counter intuitive I'm a bit baffled people are even suggesting it.
Half of the brilliance of computing moving in the way of touch is that you start to interact with digital things the way you would handle actual physical objects. People have been training themselves to use a mouse to move a cursor around a screen for the past 25 years, which is really awkward when you think about it.
With touch things work exactly how you expect them to. There have been dozens of studies with very young children who have had 0 experience with traditional computers who simply "get" touch devices, because its human nature.
I believe Apple understands this and they are not going to spend significant time solving problems that do not need to be solved, especially when said problems really just need to be unsolved and forgotten.
I agree, there is a huge difference in a touch interface and a mouse/keyboard interface. I don't think that the answer is simply to simulate one with the other.
There have been dozens of studies with very young children who have had 0 experience with traditional computers who simply "get" touch devices, because its human nature.
Case in point: my 2 year old. He can use my iPhone with no problems. He can swipe the main screen left and right to find the icon he's looking for. Start the program, and be on his way. He can also start up iTunes on it, and play his music videos that I've loaded onto it. He also completely "gets" how to use the picture viewer. Now, if iTunes is playing my music and doesn't come up with the video screen, he's lost, but for the most part, it works well. The touch interface makes so much sense that he figured out how to get around in 2 minutes.
And how is a mouse pointer different to your finger pointing at something? You know, there are even Operating Systems where the mouse pointer is a hand which you use to point the virtual index finger at things. Now, i can use my real finger to point at things, it is indeed the same f*ing thing.
This is what I always wondered about. I mean, flash is designed to operate in a keyboard/mouse environment. How would it's event model map to a multitouch model? I suspect that it could be done, but it would be awkward at best. Also, the biggest argument for flash on the iPhone/ipad has been the large amount of games already available. Well, most (if not all) expect a keyboard and mouse. There is no way they would be able to work unaltered.
You set up new events that handle low level stroke information or higher level strokes and chords. Multi-touch in Flash is possible; I was doing it three years ago on custom hardware. In some cases games would certainly need to be altered to work with the new events, but the AS3 event system is robust enough that you can write apps that work on multi-touch and non-multi-touch hardware.
There is a lot to be said for skating to where the puck will be, rather than where it is right now. For example, at one point in time most games were published on 3.5 inch magnetic floppy disks. The publishing media changed over time and that is no longer the case.
Isn't most frameworks that deals with GUI designed for keyboard/mouse?
Just because a platform allows for multi touch does not mean that the underlying functionality changes.
Multi Touch primarily gives you an ability to interact with the platform differently but the functionality it gives you access to is by and large the same.
You're missing that in order to use touch, just about every Flash game out there would have to be rewritten.
Flash and its event model are quite flexible, and there's no reason why you shouldn't be able write games for a touch based system in Flash. But you'd have to write them first, and ideally there'd be an event interface specifically for touch, or it would be overly complex.
But that applies to everything out there that relies on mouseover.
When the iPad comes along and people will be using it for more complex webbrowsing a lot of web applications/services will have to be rewritten because they rely on mouseover states.
Predominately, you're missing the same thing I missed at first, which is that SO MUCH of Flash works on mouseover -- but how do you have a mouseover event on a touch-only interface?
What I mean is that most flash games require mouseover and mousemove events to be used — or else they're completely void of interactivity — no more useful than a static image.
Most web apps, on the other hand, are fully useable without the user ever seeing any tooltip. That's what I mean by crucial. To be honest, I thought it was pretty clear already.
I understand now, originally read that as meaning the whole website. If Flash stuff can reasonably be done in non-Flash, I'd always vote for that being preferable.
I strongly disagree with the developers article. He fails to explain the simplest solution (holding the finger). This is indeed the same as moving the mouse cursor on a touchpad, so wtf..
But what i really dislike is the notion of "it is not possible, never ever, it just won't work" without objective proof why something wouldn't work.. just stating that a problem is there and based on that "fact" he goes on and on.. How many websites would be unusable with a touchpad like interace? None.
It makes me sad that this guy obviously tries to make sense of steve jobs most stupid decission of the last years, because it is clear that Apple doesn't want flash/air on the iphone not because of the interface (clearly apple would be probably the best company to resolve this issue!) but because of the threat of mobile apps and games to the appstore. First it will be games, but if i can develop my apps for adobe air for all major operating systems and smartphones.. this is bigger than most people think.
I still hope that the apple community will eventually start to revolt when flash and air applications are available for all other platforms instead of just following apples stupid business decisions with such excuses.
"First it will be games, but if i can develop my apps for adobe air for all major operating systems and smartphones.. this is bigger than most people think."
I really think it comes down to video. Video being the one thing that is easiest to replace. Many sites already do and it works like a charm. It’s a problem and it sure would be nice to watch The Daily Show but it’s down to inconvenience, not deal breaker.
If you want to make pretty much anything else work you will have to put serious work in. Just making an iPhone run Flash won’t it a pleasant experience. Some people might prefer that but there is probably nothing Apple hates more than those kinds of trade offs.
Almost nothing on the internet is "designed" for iPhones, yet nobody's complaining that Safari exists .... just like with Safari the onus is in part on the vendor to make their version suit the platform, and in part on developers and companies to fully tailor their offerings to that platform if they want.
The only difference is you're allowed to see for yourself and decide for yourself if it works well enough - if it's not in Flash anyway. Simply dismissing everything on the grounds that some things won't work is silly.
Look: Safari optimization is optional. All unoptimized webpages work. You might have to scroll a bit more than is comfortable, but in general everything works just fine.
Comparing that situation to Flash seems odd.
(If you prefer features to execution as your comment seems to suggest don’t ever – ever – buy any Apple products. Apple doesn’t think like you do.)
The only thing that seems odd is you justify a browser for the same reason you dismiss a plugin - there will be cases where Flash works fine or well enough so by your logic there should be a FlashPlayer, but there will be cases where it won't so then there shouldn't be a browser. Adobe themselves could largely ensure most Flash works just by catering specifically to the iPhone environment and automatically bridging the interface differences, as Apple did with Safari.
And it doesn't necessarily have to be a browser. There are apps that sell well and there are apps that don't, so should they just close the app store? Of course not.
"From a developer perspective, there's nothing that I needed to do to ensure that my game is functioning on this [touchscreen] tablet device. It operates the same as it does on the desktop or on this tablet device. Obviously the inputs are mapped, and that's taken care of automatically by Flash player. So Flash Player is able to bring hundreds and thousands of casual games that [are] online from day one." (Emphasis mine.)
That's just marketing speak, "functioning" is a weasel word trying to suggest, "fun to play," without actually making that claim.
He can get away with saying "inputs are mapped" because the only inputs that can be made - touch ones - are mapped, x,y co-ordinates from screen to player. Inputs which don't exist at all - hover, keyboard - don't exist, so don't need mapped.
On a related note, has anyone been able to get any interactive canvas demos to work on Mobile Safari? I've tried a few, and it seems like there are some related input issues there as well.
Yep. And that's a really simple game. Good luck to folks trying to play more complicated stuff.
The model just doesn't make sense unless Flash developers program most games' input in two completely different ways. I can't imagine that'll ever happen, but even if it did, it'd be years until then. In the meantime, every alternate-universe, Flash-having iPad user who could see an app but inexplicably could only partially operate it would blame Apple.
I think a lot of these comments are missing the point. There is a huge difference between a mouse and a finger and you cannot just abstract this difference away by pretending that hover is only used for tooltips. Flash programmers from the very beginning embraced the platform because it allowed dynamic and flowing interaction with the movement of the mouse pointer. I can think of dozens of very common Flash techniques that will never work with a finger without being re-writen.
* Combo boxes that only expose the drop handle of hover.
* Controls (video) that only appear when you move the pointer
* Camera controls where the mouse motion is used to anchor view.
* Animated buttons, flyouts etc.
This app has an "AUDIO" link in the upper right that scrolls out on mouse over to expose the actual controls. This app simply will not work in a finger environment and would have to be re-written. I would argue virtually 100% of the flash applications written will have broken functionality in a click+drag only environment.
Most of these apps will "work" but in a massively degraded state. Many of them will simply be incapable of being used.
You cannot gracefully degrade to a click/drag system from a hover/click/drag system. They are too different. Flash was developed to maximize the mouse paradigm. It will never work well with a finger interface.
I work for a company whose main site is a flash RIA. It's true that our site doesn't work out of the box on mobile flash enabled phones, although in our case it's mostly due to memory constraints. However we've created a new interface reusing most of the code we had already written for the main site, but optimized for mobile devices and without all the embedded images and memory intensive features included on the main site.
Doing that took about 2 weeks of serious dev time. Meanwhile, the native apps we're writing for each platform have taken 6 months each and still need a fair amount of work. HTML 5 is out of the question because it is missing some critical features (mainly content protection of any kind), so mobile flash is great for us.
Having had the chance to use flash enabled mobile devices on various websites, the complaints listed in this article are valid; I wouldn't use it to play flash games, but it is usable for media consumption, even YouTube's. Normal, non-mobile YouTube videos play just fine in mobile flash, for example.
I find this argument uncompelling. Nearly all actions are committed on mouse-up. This is how the default "click" events work in every environment. Click and hold becomes mouse down and hover, dragging remains hover, release can fire a mouse up, the framework can then inject an extra mouse down as part of the click if necessary. Apps that need more sophisticated behavior can ignore click/tap events and handle mouse up/down on their own.
The problem seems to be that the only event that can be passed to the page is a click, and it's sent on touch-up. Touch-down does nothing (since it could be a click or a drag), and a drag scrolls the page. The only thing they currently send through is a mouse-up, which seems to be sent through as a click.
In pseudocode:
on touch:
# do nothing yet
if drag:
scroll page
else: # was just a tap
send click to page
Granted, this is just how Mobile Safari is handling things, but how else could Mobile Safari handle things?
I thought that the problem wasn't so much detecting mouse taps, but detecting mouse hoverings. Say you hover over an item and then click on it to select. How do you emulate that click, when your finger is already pressed to the screen to drag the "mouse" around?
Sure, but who cares? There are a billion websites which use hover events, and the hover just doesn't do anything on the iPhone. The author acts like this is some kind of NP complete problem or something. It's at most minor revisions.
I care. Personally, I would rather that something not work right off the bat, then think it will work, only to find out that it won't actually work, that by design, that if it was to be released right now, it would be impossible for it to work on the device that I'm on a decent portion of the time.
Don't get me wrong; I'm not saying that the problem can't be solved. I'm just questioning the wisdom of solving it, in the first place. Or maybe how well the solution will work. But, not the solvability of the problem itself - especially if a few smart people are thrown at the problem.
This might be the part of me that likes my nice and pretty Mac-specific software, but, I don't see anything wrong with software that targets one specific platform. And for Adobe, with Flash, that platform looks like it should be the desktop, not the mobile web.
Consider the embedded Flash people use on the web. Many Flash games (most?) critically rely on mouse position. Many Flash video players hide controls until the applet is hovered over.
In contrast, the usage of hover in HTML pages is almost always just to highlight something to show that it can be interacted with.
Maybe I'm not reading the posting right, but I really don't see a problem. Why can't the screen detect a slide over as a mouseover, but consider a tap as a click. Current trackpads already do this on laptops. I can drag my finger around to move the cursor and trigger a click by tapping on the pad.
If my laptop's trackpad can do it, I see no reason why a touchscreen can't do it.
Because a phone has no concept of a cursor. When you lift your finger off of the phone, it is gone. Motion on a trackpad is saved by the computer as a change to the cursor position. This can be simulated by saving state in the touchDown/touchUp events, but it isn't like polling for the current mouse position.
The other thing to think about is that if you do have a mouseover event, you are likely going to try and display something (like a tooltip). On a phone, this would be covered by your finger.
I don't understand why it doesn't have a concept of a cursor? Windows Mobile phones have the concept of cursor and I can have a mouseover by holding the cursor over something whether it's with a finger or stylus, and then click by tapping either with my finger or stylus.
What's the disadvantage to having an invisible cursor so that mouseovers work? Couldn't the cursor "appear" to the system when your finger is on the screen, similar to how the cursor shows for restaurant POS terminals when your finger is on the screen and then disappears when your finger is off it. Clearly I'm missing something.
What's the reason for the downvoting of this and my follow-up post? This is very disconcerting, it discourages me from participating or contributing to an active conversation. HN etiquette requires a reason for downvoting.
As a long time member this is very frustrating. I'm sure I'll get even more downvotes for pointing this out.
Exactly. When I was reading this, the first thing that came to my mind was that my MacBook has a multi-touch trackpad. Probably the solution would be to enable a mouse pointer over flash content.
But gaming is a much lesser experience with a trackpad then it is with mouse and keyboard; my Bloonz scores with and without a mouse prove it. Also, this doesn't get around the many games that require a keyboard.
I don't think the author is aware of Adobe's Slider framework which addresses the most basic needs (mobile device gestures). Beyond that, many of us are fully aware of the challenges of multitouch & gesture and are experimenting with variable-input interfaces (mouse, gesture, multitouch, and button/arrow based remotes).
The impression I got from (briefly) reading the Adobe Labs page on slider is that it still would require all existing content to be re-written for it... which really negates the main reason for Flash on the Ipad: the dearth of existing Flash content on the web.
Slider will require a different front end, but you don't need to toss the entire code base.
As for existing code, yes that will be a problem. But most Flash developers think of projects as evolving and/or disposable. Most of us aren't locked into stuff we did 3, 5, or even 10 years ago. We move forward and enjoy finding solutions to new challenges.
Maybe for your own site, yeah. But for sites that you've built and handed off? Are you going to update them for free? Are the clients going to pay you (again) to rework the interface (non-trivial) for a small minority of users?
I will agree that the author has done a good job pointing out that many current Flash apps may have problems on the iPad. Many of those would be addressed if the blockade was lifted, but certainly not all.
You can simulate a mouse with existing hardware by decoupling the moving of the pointer and tapping (=clicking).
Therefore, moving your finger around the screen or tapping in a new position would only move the "mouse" pointer and not generate mouse clicks.
The mouse buttons could be mapped to some physical buttons on the device or to virtual buttons on the bottom or corners of the screen (think of your typical laptop touchpad layout).
Even better, the software could support a touchpad mode where the finger coordinates on the screen aren't mapped 1:1 to where the "mouse" pointer moves but as deltas similarly to how a touchpad works. This would allow for a fine-grained mouse movement even with the touchscreen and this would naturally limit the mouse clicking to specific locations / buttons only.
This could be done and applications wouldn't have to be rewritten. Likewise, you can just run a desktop OS and desktop apps on a tablet and nothing would have to be rewritten. Everybody thinks that sounds great; once they use it, nobody thinks so anymore. Because it's awful. That's the whole point of iPad using an OS and a collection of apps that from day 1 has been intended for touch.
Since you can do that selectively in software there's no reason Flash shouldn't do it while the tablet OS and other applications should.
Touchscreen UI is all good but there will always be a class of applications that want both more precision and mousemove events.
As a personal note, touchscreens are painful to use unless you only have functionality behind big baby-size buttons. I wish I could always switch to such a touchpad mode if I wanted to.
It seems to me that most of the 'mouseover' problems described can be handled with click-and-hold.
Nope; the "Tap and hold" gesture is used by the Safari (and the iPhone OS, for that matter) for other tasks already, namely to bring up the cut/copy/paste element.
There is no reason that I can think of that just because Mobile Safari uses tap and hold for some functions, that it couldn't pass it along to Flash when performed on a Flash element.
Changing defaults around means that you're forcing people to learn new behavior specifically for your site. Not a particularly user-friendly thing to do.
And in Safari, doing this would raise even more confusion; the default behavior works in other parts of the app (the next tab over, for example) but not the tab for your site. It is easy to see a user getting confused as to why something isn't working for one particular site. Maybe they want to copy text from a flash element, but it ends up doing something completely different.
No, I would expect the software to know what is a Flash element and what isn't. I would also expect the software to make it easy for the user to know one from the other and see the behavior is different.
What do you mean? I don't recall ever using Safari on iPhone and having tap-and-hold not do the Safari default thing (pop up contextually-dependent menus giving choices like "Copy" and "Open in New Window" for links, "Save" and "Copy" for images, etc) and don't think that JavaScript has access to that.
The closest thing that I can think of is -webkit-user-select, although I'm not sure if thats available on the iPhone. Nor would it let you change UIMenuController behavior. I suspect that if it is possible, it would be under an NDA. That is, after all, how these things seem to work. Would love to be wrong though.
Alternate, Occams Razor take: Apple just hardcodes the options, based on the type of object that is selected.
A user would still have to understand why trying to grab+drag to scroll would work on one part of a page but inexplicably not on another part (eg, a Flash ad).
Most likely it would be a crap shoot between pages that took the iPhone/iPad into account, and those that would confusingly not work.
It would be cool if a touchscreen device could fire a mouseover event when a finger was NEAR the screen, and only fire a mousedown event when the finger TOUCHES the screen.
If this is true of flash, why is it not the same for Javascript. If suckerfish style menus can work on iphone, why would hover and clicks be handled differently?
Is that JavaScript or is it just CSS? If a HTML element has a CSS hover property, the first tap will activate that instead of the anchor. But, in any case, if it works for CSS it should play well with others, too.
"Current Flash sites could never be made work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware."
I can think of a few easy fixes. A toggle button that makes touches count as hovers rather than clicks (easy). Hardware that detects a finger hovered near the screen (probably hard). Nobody caring because most things that occur on hover you can do without, and websurfing on even the best phone is so fraught with compromises and frustrating relative to a desktop that you really don't care (free).
"Nobody caring because most things that occur on hover you can do without,"
I don't think that's at all true. Yes, the concept of mouseovers could be done without (if it had been done without from the very beginning), but since so many Flash apps rely on it now, you can't have a decent Flash experience on a device that doesn't have it. Try using many Flash video sites without an ability to mouseover: the controls hide until their moused-over. A click on anything but the controls means "pause".
"and websurfing on even the best phone is so fraught with compromises and frustrating relative to a desktop that you really don't care (free)."
I strongly disagree. The web on phones before iPhone was indeed compromised and frustrating. But iPhone and some other recent phones have such good browsers that one often hears of people saying they reach for their phone more often than their laptop.
This seems ridiculous to me, it's almost as if the thought of using say... some sort of User-Agent to identify and differentiate the experience of the software is just well too simple to consider?
"A) The best case: every Flash app on every site is re-thought by its designers and re-coded by its programmers (if they’re even still available), just for touchscreens. They wouldn’t use mouseovers any more—or else they’d have dual versions of all Flash content, so that mouse users could still benefit from the mouseovers they are used to. That’s a ton of work across the Web, for thousands of parties, and just isn’t going to happen. Plus, with many sites, mouseovers are so fundamental that the very concept of the site would be altered, creating a whole different experience that would annoy and confuse the site’s existing users. (And would this be any easier than simply re-designing without Flash at all? Not always.)"
You'd still need to re-implement a significant part of each and every Flash app/game. Many of them would require a totally new UI and interaction model.
You can tell from this article that the programmer thinks universally and in so doing, excludes a vast area of potential, especially in the future and more specifically in what he notes as "B) Gestures, finger gymnastics". You can stick your one finger up your nose but with more than one finger, you can do a lot more. I for one am ready for more complex interface functionality. I thought this was more obvious though perhaps it should be clearly stated: Get used to the growth in complex multi-touch gestures. More to come soon.
My guess (just a wild one for not being a thinker of touch interface design), would be to solve the solution by first detecting what kind of flash app it is and how much area it takes up, then anytime the finger scrolls over that particular area, there would be a cursor (the user could turn this off, but this is what would happen by default). As the cursor is led by your one finger which never lets up, meanwhile 1,2, 3 or 4 taps with another finger along with other more complex combinations eventually could perform various flash actions. Contrary to Dilger's primary thesis, I believe it's pretty straight forward that Apple is not as limited in thinking to make it happen if they want to, and that even the general audience is ready to take another small step in the gestures department.
I think my brain just exploded from all that complexity :)
Why should Apple think hard and work hard in order to solve a really hard problem they have no particular interest in solving?
Maybe there is an elegant solution out there to make all or nearly all existing Flash apps work, but it won’t be easy and I don’t see Apple investing much in technology they clearly think is dying. They normally never do.
I agree, Apple is prob not going to do it, but the problem is not "absurd" or "impossible". That's my main point. The author of this article gives up and says it can't be done. I disagree with that. I gave one of many possibilities. I prefaced it by saying it was wild guess, but again, I don't think it's impossible.
Actually, Flash supports both multitouch and gestures which are separate from mouse events, thus it's possible they may work concurrently (i.e. in the same app with common mouse input).
Flash has not supported multitouch until the latest release, meaning no existing flash content was developed with that in mind. Nice logical fallacy though!
You misread my statement which was part of the "how would you solve the problem" thread.
My point was that now that Flash does support multitouch and gestures, it's possible to add those features to existing content (yes, this will happen). Naturally it won't apply to all projects but it's far from the sinking ship that the author suggests and rabid HTML5 supporters are praying for.
I hate the idea of Flash on an HTML5 appliance, but the article and (as of 10 AM ET) the other posts in this thread miss one obvious way of supporting this with no extra buttons/toggles and no double clicks, click-hold-clicks, or other unusual behaviors.
If a user touches a Flash zone, show a cursor, and have the cursor moved by the iPhone's accelerometer. Tap to click.
For elements such as the video player example, the cursor (just as on PC) can fade after time, and a tap or shake would re-show it.
For most of the scenarios he mentions, using accelerometer to drive a hovered cursor would be a particularly intuitive solution.
Don't have any special insight into flash but I do know that there certainly seem to be issues with playing flash on my mac (with any browser) vs PC. I'm guessing it has to do with GPU vs CPU issues but whenever I play a video (via flash) my laptop fans go on full blast (probably because the CPU is in overdrive). Quicktime video never seems to have this problem.
Flash also tends to have playback and interaction issues.
Half of the brilliance of computing moving in the way of touch is that you start to interact with digital things the way you would handle actual physical objects. People have been training themselves to use a mouse to move a cursor around a screen for the past 25 years, which is really awkward when you think about it.
With touch things work exactly how you expect them to. There have been dozens of studies with very young children who have had 0 experience with traditional computers who simply "get" touch devices, because its human nature.
I believe Apple understands this and they are not going to spend significant time solving problems that do not need to be solved, especially when said problems really just need to be unsolved and forgotten.