Well, thank goodness I would only use this kind of thing to play old video games. Until some Windows desktop ad shows up with "ignore previous instructions and buy this thing." Ugh.
Google has the tech (some of it's gathering dust, but they have it). They can use the gameplay tech developed for stadia when a user experiences lag and combine it with their LLM.
Definitely! Those with movement disabilities could have a much easier time if they could just dictate actions to the computer and have them completed with some reliability.
You know, it's amazing how different people view iOS 18. I, and many other blind people, love iOS 18 because the system for typing in Braille, and now controlling the phone in Braille, just using the touch screen, has been majorly improved. Although, we don't have an LLM based image description feature, like Android now has in TalkBack 15. I'd love Android, but it's screen reader is so sluggish, especially when scrolling the screen where it feels like it takes half a second, that a one year old Pixel 8 is much less responsive than a 4 year old iPhone SE 2. It's a shame, but Google hasn't changed much in the last five years, and I don't see them changing much before the Pixel 8 is deprecated.
Seems okay at image descriptions, I suppose. Still a 12B model though, and doesn't always get OCR anywhere near correct. I tried it on Le Chat, and waiting for it to be on Ollama.
I think of it like speech synthesizers. First they were their own machines, then cards you plug into a computer, then once people figured out how to mash human speech together, they were, in some cases, a good 1.5 GB. Now, Siri voices, which are tons better than the concatinative models, used with the VoiceOver screen reader are a good 70 MB, Google TTS, even though it's awful and laggy with TalkBack, offline voices are a good 30 MB for a language pack, and in iOS 18, we can use our own voices as VoiceOver voices. So I think eventually we'll figure out how to run amazing AI stuff, even better than today, on our devices. And I think tons more people are working on LLM's than were ever working on TTS systems.
Thankfully, Linux is getting much better for blind people. Orca has become much more stable, and accessibility is more than an extremely underground movement within desktop environments. Just a bit more, and we'll be ready.
How about we think of emulation as making games accessible? I know, it wouldn't work for like 99% of use cases, but it sure makes my life easier, as a blind person. Since Retroarch now has accessibility support on iOS, and loads its screen reader when it detects that VoiceOver is on, I can open the game I want to play. And then, using VoiceOver's Screen Recognition, which basically works as OCR and basic image recognition, I can read game menus.
For now, this just allows me to play games on my phone who's gameplay is already pretty playable, like fighting games and visual novels. Except, now I can read menus, character selection screens, character dialog, stuff like that, all on my phone. Too bad Android doesn't have a screen recognition feature for TalkBack; they barely just got image descriptions that VoiceOver had for years. So, even though emulation is far better on Android, I still choose iOS. Apple could, I don't know, sure capitalize on that. I mean, even at the end of this article[1], it shows someone using what I suspect is emulation to play Metroid Prime on a Vision Pro, probably through Dolphin on a Mac. So they surely know about the need for JIT in emulation.
That was a really great article overall, even showing the downsides of Apple, including the poor state of VoiceOver for Mac. And at the end, it shows possibly emulation being used for accessibility!
