I saw a demo a while ago where they used Infrared makeup on someone, and used a sponge to make it really splotchy. The makeup doesn't show up in Red, Green, Blue wavelengths.
So, they could simultaneously capture RGB, and IR. Using multiple cameras on the IR, and the splotchy makeup, they were able to capture thousands and thousands of points, and get color info at the same time.
Seems like the right approach to me. I could totally see a Kinnect-like device taking advantage of this, allowing someone to do facial capture like this at home.
The infrared makeup (pretty sure the only company that kinda pursued that path was Mova) was kind of unwieldy and it's kind of fallen out of use already. Most facial capture in the future will probably just utilize some kind of depth capture (light field, structured light) in concert with a normal witness camera to reconstruct facial features.
I believe that was for The Curious Case of Benjamin Button, which was pretty unusual. See if you can find e.g. the motion capture for Pirates of the Carribean. But you're right that the IR capture seems to make more sense.
You're probably talking about Mova. Facial motion capture (of that quality) is very different from full body motion capture since the face has so many more degrees of freedom. So generally it is done in an extremely controlled environment with many simultaneous cameras and specific lighting and very little head movement.
The problem with doing high quality real time performance capture at home is you can't control the lighting or occlusion - sometimes even a human looking at the footage won't be able to discern what's going on. Not to say it won't ever happen but it will require extremely significant advances in computer vision.
I saw a demo a while ago where they used Infrared makeup on someone, and used a sponge to make it really splotchy. The makeup doesn't show up in Red, Green, Blue wavelengths.
So, they could simultaneously capture RGB, and IR. Using multiple cameras on the IR, and the splotchy makeup, they were able to capture thousands and thousands of points, and get color info at the same time.
Seems like the right approach to me. I could totally see a Kinnect-like device taking advantage of this, allowing someone to do facial capture like this at home.