That is really cool! They are using a realistic 3D model of the persons face and eye-tracking in the VR headset to make a good guess at their facial expression, effectively reconstructing the face that's hidden by the headset.
Not something you can easily do as a consumer yet, but for high quality mixed reality production this seems like a great boon.
I really liked the approach that Oculus are taking and which Zuck demo'd last year[1], with cartoon avatars that are still expressive enough to convey facial expressions.
It definitely falls on the other side of the uncanny valley, and anyway, who is going to want to render a realistic version of themselves into the VR world, when you can be <insert favourite person/creature/character>?
That said the tech in the OP is very cool, and I can see plenty of use cases (e.g. business meetings or celebrity broadcasts).
In the FB video, I think the facial expressions were like emoticons: they had to be manually invoked. This approach could do it automatically in realtime, by mapping the facial movements onto a cartoon avatar (probably easier than onto the real face).
Interesting, do you have a source for that info, or is that a guess?
Watching the video the eyebrow movements look a bit twitchy, I'd assumed they were face-tracking them. I have no supporting evidence beyond watching the video, so I'd be interested to know more details either way.
Probably fails the cost-benefit analysis for most people doing Mixed Reality videos for their VR products. However, if it was productised, that'd be a different story.
You should probably return to reddit. There's a standard for commenting on Hacker News. I don't make the rules, but whining about people being "butthurt" because your low-effort comment was downvoted definitely doesn't fit with the general vibe of the place.
“We’re at the beginning of this. Vive is the most expensive device on the market. It’s barely capable of doing a marginally adequate job of delivering a VR experience. We have to figure out all sorts of other problems before even the hardware question gets answered, much less what’s going to be the compelling content.”
“If you took the existing VR systems and made them 80 percent cheaper, that’s still not a huge market. There’s still not a really incredibly compelling reason for people to spend 20 hours a day in VR”, Newell said. “Once you’ve got something that will really cause millions of people to be excited about VR, then you start worrying about cost reductions. […] Some people have got attention by going out and saying there’ll be millions of [VR unit sales] and we’re like, wow, I don’t think so. I can’t point to a single piece of content that would cause millions of people to justify changing their home computing.”
Not forgetting to mention... "Plus, it just makes you look like a dork" - Gabe Newell /s
Some people, tsk tsk - they have no sense of humor.
Not something you can easily do as a consumer yet, but for high quality mixed reality production this seems like a great boon.
For an explanation of mixed reality videos are made I recommend this: https://www.youtube.com/watch?v=8T7ux3DXP_w