This project is from 2019. The tools to do this are now more widely available and the realism has advanced considerably since then. This project was an early example that illustrated to a wide audience that audio and video are no longer reliable sources of truth. By comparison, the techniques used in Forrest Gump (1994) look crude.
Saying this universally leads to a too-rosy view of the future, I think, and downplays the amount of hard work needed to maintain society.
Consider advances in travel - you basically had people who saw, within their lifetime, a world of trains become one of planes, including some supersonic passenger flights (not to mention going to space). Even bullet trains are old at this point.
Lotta other examples of stagnation in the physical world rather than the information one since then.
I'd love to take a bullet train from SF to LA. I make the trip every two or three months for work and just hoping on a train and zoning out would be far nicer than running through buses/light rail/airport security/cramped airplane/etc.
Distance would be pretty similar to the Tokyo to Osaka leg of the Tokaido shinkansen. Total time would probably be similar. I left my apartment 3.5 hours before I arrive in SoCal. Hour and a half on public transit, 30 minute buffer for flight, 30 minute boarding time, 60 minute flight.
The authors should redo the video every couple of years as the tech progresses. It is easy to spot the fakery in the 2019 edition, but I'm sure it's harder now.
1) Multiple camera angles. A simultaneous left / front / right view of a person speaking would be much harder to fake.
2) Higher-resolution video and high-quality audio are harder to fake, although this might not be a problem for entities with sufficient resources and motivation.
3) Attribution to a real person. While this raises problems for people who want to remain anonymous due to very reasonable fears of retaliation, it also makes it harder to push fakes out to social media.
Note many fake 'breaking news' videos are just stiched together old footage, no AI deepfake required. This has been noticeable across multiple platforms wrt Ukraine, for example:
Unfortunately, while those are good ideas, they’re also cat and mouse games. It’s a bit like saying that client side anti-cheat software makes it harder to cheat in videogames. It does, but you’re looking for an unbeatable test, which doesn’t exist.
Nothing exemplified this problem more than the recent Twitter takeover. On tiktok, fake news was running rampant, and I saw it happen in real time. At one point someone said that a Twitter employee banned Elon on his way out, while showing a screenshot of the “eIon musk” fake account: https://www.tiktok.com/t/ZTRx6G9mG/
There were also actual deepfakes of Elon’s voice that fooled me. I wish I saved it, since it’s highly relevant to this thread. But it only became clear it was a fake when it started saying ridiculous things. They even had a video of Elon talking, and they didn’t have to bother syncing his lips with the audio. Viral clips just need a veneer of authenticity to short circuit our monkey brains.
It wasn’t till I fell for that deepfake that I started realizing how much of a problem we might be facing. The next few presidential candidates might need to put out statements saying “X wasn’t true,” while the majority of people already believe X because they saw it happen live on tiktok.
Yeah, that's all true. This might lead to a revival of professional journalism and actual reporting, however - i.e. if a known reporter with a good track record writes, 'today this presidential candidate said X, and I heard it myself', then that might become a more reliable information source than any audio or video clip.
Ah, but what happens when a deepfake video of that professional journalist starts going round saying 'today this presidential candidate said X, and I heard it myself' that matches the previous deepfake?
I guess the counter (which I think you're pointing out) to that is to publish text to a known-trusted website rather than a video, but I wonder how many folks who get their "news" from TikTok would read it.
Uhm, I am not sure I get the relationship: if anyone can get verified, that's only a win.
By "verified", I assume Twitter has verified that their identity on Twitter matches their real life identity, and I don't see a problem with anyone having an ability to claim that. What's wrong with Jane Random being verified that she _is_ Jane Random?
If people are basing their trust not on someone's identity and history, but on their celebrity (blue mark so far), that's where the problem is.
The problem arises where an account gets verified as one thing and then changes to become another, this happens all the time with trolls who can make/buy/extort verified accounts.
They then spoof being the real person, and verification then means absolutely nothing cause they are functionally identical to a bypasser.
By having verification be really hard to get, trolls have a lot harder time getting accounts and the problem is smaller in scope.
Allowing identity changes without requiring reverification to keep the "verified" status sounds like a problem with an obvious solution: Require revalidation, or remove the mark.
Having initial validation accessible is not the issue here.
Ah, then I was got by that fake twitter ban joke and didn't realize it till now. In my defense, it's a cute little meaningless stunt that I'd like to imagine myself doing in similar circumstances, though upon a bit of thought it seems unlikely since it could cost you a severance that might be pretty valuable - there a plenty of strings attached to those to keep the severees in line.
There's been a phenomenon of verified accounts basically doing ""suicide by cop"": changing their display name and profile pic to match Elon Musk and then doing a bunch of joke tweets as him, which are great fun until they are caught and banned.
Video integrity is an idea we have to let go of. Audio is robustly fakeable at this point, video is coming up to the threshold fast.
What we can have is attestation and reputation, then provenance.
It starts with a container format that allows signatures of both the entire video with audio, audio only, and video only, along with reasonable segments (three seconds?) of each of these.
Excerpts of any of these can be transcluded into another such container. The editor can transform it in any way, the resulting display doesn't have to match the particulars of the excerpt (captioning, noise filtering, color correction) but the excerpt is itself included, or at least the signature.
It would be useful for some people to have cameras which do this at the moment of recording, along with GPS, cell location data, and so on, to get really bulletproof documentary footage.
I don't want that built into my phone, necessarily. As a container format and ffmpeg style tool, this would support anonymous attestation which can be demonstrated by the keyholder at any future time. As a built-in subpoena-able feature of my phone or mirrorless camera? Hmm, probably don't want that. If it snitches proactively, not a chance in hell.
Building attestation into hardware sounds like a good idea in theory, but doesn't actually work. All it tells you is "this is what the camera saw", it doesn't actually tell you that what the camera saw was real. You can just point your camera at a high resolution screen playing a doctored video, and so sign whatever doctored content you like. Or take a video of a real faked scene with actors.
We mustn't confuse attestation with reputation, nor even the combination of attestation and reputation with truth.
Attestation is nothing more than someone staking their reputation on the veracity of what they're sharing with the world.
If I'm being frank, I thought this was too obvious to need stressing. If the camera is pointed at David Copperfield, cryptography can't make the lens show you reality.
Attestation solves a lot of problems, but like my first sentence said, video integrity isn't a problem which can be solved anymore, if it ever was.
The trusted GPS problem might be solved by actually turning GPS upside down and having the device emit the signals instead of receiving them—it’s known as reverse or inverted GPS.
One way to get a verifiable time stamp would be to show knowledge of an unpredictable but well-known thing in the past (providing a lower bound), then to create the same thing yourself (providing an upper bound). I wrote about it here: https://news.ycombinator.com/item?id=33558566.
Your comment in particular made me feel like it was worth posting!
The founder of Bellingcat discusses this in his recent book.
He mentions things like "where/when did this happen", where were the people/things in the video at that time, which source did this arise from. Basically there's a whole web of context to any event, which is why the breaking news is the best place for this, but the long term viability is much harder.
He also mentions some NYT tech for timestamping their videos as they record them and to detect manipulation.
> What is the Coalition for Content Provenance and Authenticity (C2PA)?
> The C2PA is a Joint Development Foundation project to collectively build an end-to-end open technical standard to provide publishers, creators, and consumers with opt-in, flexible ways to understand the authenticity and provenance of different types of media. C2PA opposes efforts to make content provenance measures mandatory.
It would require hardware changes, but we already have all the required cryptographic primitives to mostly mitigate these concerns.
Trusted timestamping can attest that a given piece of media was not created after a given instance in time.
Trusted hardware running trusted code, combined with remote attestation, can offer a high-degree of assurance that a given video clip is not a product of CGI/ML.
Digital signatures by the entity responsible for a video clip's capture/creation mitigates the creation of media not sanctioned by that entity. Subjects in a particular piece of media could additionally apply their digital signatures to further increase trust in its authenticity. (This is effectively a cryptographically enforced version of your third point.)
This isn't a silver bullet, and suffers all the pitfalls normally encountered in the implementation of cryptographic schemes.
Your thoughts are very interesting, but I'm afraid remote attestation may not be enough. Processor instructions aren't sufficient as verification. Suppose I use AI to create a deepfake, then open up my phone, grab the wire which goes from the camera to the microprocessor, and rewire it so it goes from an Arduino to the microprocessor. Then I program the Arduino to send a signal to the microprocessor just like the one my camera would send to the microprocessor, had the camera just taken my deepfake as a photo.
I suppose a chip inside of the camera (in a tamper-resistant housing that self-destructs if you attempt disassembly) which signs the photo before sending it to the microprocessor could be sufficient in practice. Especially if the provider of the key infrastructure embeds a separate key in every camera, and monitors for abuse.
Anyway who is gonna build this? Perhaps Google could start selling special cameras to journalists, such that if the photos in a story on Google News were taken using that camera, the story in Google News gets a special checkmark next to it?
It wasn't elaborated on in the original comment, but yes you're absolutely correct, the imaging sensor itself would also need to be part of the cryptographic chain-of-trust. This is likely the most crucial part of such a system, as all the other hardware aspects are already starting to become generally available.
> Anyway who is gonna build this?
If and when specialized imaging sensors become available, I don't see this being a huge lift. One wouldn't even need purpose-built cameras for all but the most high-assurance recording requirements. Remote attestation and trusted code execution is already being incorporated into most newer devices. Bringing it all together would mostly be a software problem at that point.
Of course, there would also need to be a value proposition to justify any kind of work towards this. I personally see this as an inevitability given the frightening pace of progress in the ML field, and the increasing public concern over things like deepfakes.
You could start a startup building the specialized imaging sensors, with the aim of being acquired by $bigtech so they can tell congress "yes we are doing something about deepfakes" and eventually tell consumers "you can tell that photos taken by our devices/posted to our site are real, the other sites are full of BS"
I don't see how it all comes together without a partnership between the tech companies that display media (Youtube, Facebook, Twitter, etc.) and the tech companies that provide hardware (Apple, Google, Samsung, etc.) Maybe what's needed is to go through a nonprofit like the Partnership on AI.
Google is an interesting target because they are basically the only company that's both a media displayer and a hardware provider. If Android devices get a "verified non-deepfake" capability before Apple devices get it, I could see that as being a significant leg up for Android actually. It encroaches on Apple's traditional branding of "we make products for important people". I wonder if the Titan M has capabilities which could be leveraged. https://android-developers.googleblog.com/2018/10/building-t...
Google would probably sell loads of Pixel phones if it upranked "verified non-deepfake" Youtube videos recorded using Pixel cameras & microphones. Every aspiring Youtube star would know to record using a Pixel. The deepfake thing is a pretty good excuse to defend against an antitrust lawsuit, especially if Google can say that the capability will eventually roll out to competing hardware.
Fun fact: the original Apollo TV broadcast from the surface of the moon was "slow scan", with a different resolution and framerate to broadcast TV. The conversion process was to display it on a suitable slow-scan TV (long retention phosphor) and .. point a terrestrial camera at the TV.
The key difference is that digital signatures allow revocation-of-trust to be an effortless, automatic and delegatable process. As opposed to the current process that involves individual (and intrinsically limited) knowledge of who can be trusted. Frequently one doesn't even have provenance information for a media item, which is a prerequisite for making a trustworthiness judgment call.
The "hocus pocus" is literally the point that distinguishes this scheme from the plainly non-working system that we have today.
Just because something is different from something that doesn't work doesn't mean the different thing will work: most things don't work.
I'm not seeing anything in your description that couldn't also be said about using PKI to allow people to sign stuff they published. (And better: it's not broken by non-deceptive edits, like cropping or brightness adjustments)-- except that PKI already exists so we can't imagine that it would magically solve these problems. :) (and there is a long long history of people thinking varrious PKI ideas will magically solve varrious problems and then failing to solve them at all)
Trusted by whatever certification entities (ideally multiple ones, from non-aligned states) decide to deem a certain piece of hardware as "trusted". And then by extension, by any individual that trusts those particular certification authorities.
Yes, I'm aware this leaves the door open for subversion of certification authorities by nation-states. It's still better than the nothing we have today.
Cryptography. Honestly, all sources (news outlets, self publishing celebrities, etc.) should start cryptographically signing their content one way or another. Including video and audio. (I haven't looked up whether there are existing, easy to use/practical schemes for this, but I guess there are and no need to reinvent the wheel. Even if the actual tools are missing.)
An animator used the voice cloning tech to make a "leaked episode" of a cartoon that ended years ago: https://youtu.be/shp9jzJ4MOw
It's funny how both that and this used "low sound quality" to mask the imperfections in the cloned voice. But I suppose it's only a matter of time before that's no longer necessary.
Though I must say that if the intention of the video was to make me doubt abilities to spot deep fakes, it didn't manage to achieve its goal. Both Nixon's face movements and voice felt highly unnatural and robotic to me. Although I'm sure it won't be many years until the results of such experiments will be come indistinguishable, we are not quite there yet.
His face movements were excellent, they were much like Nixon's real mannerisms. The problem was the edges of his face were pretty bad. The voice was also bad, with obvious ghosting and glitches.
That said as someone else pointed out this was done in 2019. If this was made today the face edges would be much better and audio clearer. I'd love to see this remade now.
It's interesting compare it to a real speech. Nixon has weird mannerisms but the deep fake feels like a disconnect between the speech and the words. It's tough to describe how my brain differentiates weird mannerisms and looking odd in a more fundamental way. In the deep fake when he reads off the names of the astronauts his shakes in this weird way that mind immediately screams that's not human, but nothing feels like that in the checkers speech.
I enjoyed it, although not as much as most critics. I found it took a few episodes to hit its stride, and enjoyed the first and second seasons. At its heart it's a character drama, so if you're watching it only for the science interesting moments are few and far between.
Later episodes fall victim to lazy "a major catastrophe is about to occur, engineering team comes up with moonshot solution just in the nick of time to save the day" more often than I'd like. And the lesbian romance storyline seemed forced. I could barely finish the third season as it really went off the rails.
The aleida character is the worst though. She somehow has super powers that no one else posseses, but they never articulate her thought process or how she does it. It's just: entire team of rocket scientists stare at problem fruitlessly. Aleida stares at problem, inspiring music plays, her eyes light up and she tells you the solution. And then she explains simple things like impulse vs momentum to other rocket scientists, as if they didn't understand it before.
That's how it rolled for me, as well. That first season was perfect as a "what if" exploration. Then it like if ran out of gas quick and grew much less plausible...
The parts I loved about it faded away, but I stuck around anyway. :P
The majority of people and critics reviewing it disagree with you as do I :). Personal preferences aside, the recreated Nixon footage and conversations are well done and quite convincing, which was the point of watching it I believe.
The show’s premise is a nuanced, optimistic take on what humanity might have achieved had we continued the space race. It focuses on historical plausibility, a bit like The West Wing. The main characters “pay the price” for this early progress in the way you imagine a hero astronaut/cosmonaut might - with their lives and/or livelihoods.
I watched it with my dad however and he wouldn’t shut up about “wokeness” (we disagree on a few things). The show is effectively an imagining of “America, made great in the first place”, and at least the core premise - that we lost our hunger for exploration after winning the space race, _used to be_ a bipartisan talking point.
Absolutely recommend it. Also, the imagined space shuttles they get to come up with are amazing.
> I watched it with my dad however and he wouldn’t shut up about “wokeness”
I thought it was leaning a bit too heavily on what we consider modern issues. But then it turns out that some of the characters in the story where real and lived through this or something close to this (I forget now, but one of the women astronauts that never got to fly was a real person)
It should be considered legitimate to call out agenda seeking, even if you agree with the opinions of the agenda-holder. When creators sprinkle their convictions opportunistically, people will notice that they're being told what to think. For many, that leaves a bitter taste, even if they agree.
There are a few stories were currently popular agendas overlap with reality, such as the story of Alan Turing. You cannot tell his story without also talking about the government abuse based on his sexuality. In those situations, you can tell the story neutrally, and let the audience judge. Obviously fiction is different though.
The audio and video of Nixon was obviously fake. I knew the text of the speech was real since it was released years ago and though I couldn't remember all of it, I do remember how he tried to set the narrative of heroes lost.
I was a kid back then and I watched the TV broadcasts as they happened or as they were first reported so I was already familiar with those short videos though I originally saw them all on a black and white television. I have seen the same footage in color a number of times over the years since.
Growing up during this time period there was a sense of wonder, awe, hope, and we could look forward to even more missions and discoveries. It was a great time to be an American and there was some enhanced prestige associated with being a Texan since a lot of this happened because NASA was in Houston. Texas schools focused on science, technology, mathematics, and kids fitness was still encouraged through programs like the President's Physical Fitness tests. Even today I weigh only 30 lbs (13.6 kg, 2.14 stone) more than I weighed when I graduated high school. That part of the message stuck with me.
The video had too many repeat sections that were obvious fakes since it was too jerky. Live video and audio never sounded or looked that bad even when you were watching with a tuned rabbit ear antenna.
I have a feeling that the audio portion of the astronaut dialog designed to lead the viewer to believe that they had seen bad things happen was fake too and it used manipulated footage to build the mental image.
It was good seeing Walter Cronkite again. We watched Chet Huntley and David Brinkley a lot too. I think they were NBC news. Journalism has sure fallen on hard times over the last 40 years after we (the hearts and minds of the American people) were sold out to corporate pigs by a change in regulations that allowed one corporate entity to control all forms of media available locally. Independent local newsrooms could not compete with a well-funded national operation that could pump news stories into your homes over television, radio, and through a competing newspaper. Local newspapers sold out to national interests who used (and still use) partisan editorial control to sell their narrative. If you are under 40 your news has been a feed of carefully curated stories delivered by pretty faces intended to keep you engaged. Using those pretty faces to create a facade of attractive people with nearly plastic features evidently works. One network in particular uses so much makeup even on the male personnel that could all be wearing silicone masks during the broadcasts. This is intended to distract you from the message and to keep you watching their channel. Remember that Huntley, Cronkite, and other journalists on television every night weren't chosen for their looks. They were pretty ordinary looking men who would probably be a 5/10 where today's anchors are gussied up to try to hit an 8/10 appearance. You won't see any two-baggers on TV nowadays.
Unfortunately, all this new deep fake technology will be convincing enough to fool a lot of people. Propaganda is a huge problem today in all media forms without resorting to totally faked broadcasts delivering invented "truths". Our educational system is failing the people of the United States in many areas as religion goes back into classrooms, charter schools rob the public school systems, and private schools increasingly become a pathway for wealthier parents to pass their own political, religious, or social beliefs on to their kids with the assistance of others in their social circles. It is a tool for siloing the next generation so that they have none of the normal social interactions with people from different backgrounds that they would have had in a public school setting. This helps the older generation maintain social problems like racism, homophobia (through some religious-based private schools), women's place in society, etc.
A long time ago I thought we were getting close to solving problems like racism. As an older adult I see that we lost our way, not because it is intractable but because we were intentionally led back to the well-traveled path by bad actors after we had initially collectively chosen the less-traveled branch which offered a solution. Picking the less-traveled branch of the trail required us to be courageous, to be open-minded, empathetic, and civil to each other. Instead, while navigating this difficult path we chose leaders who did not deserve our confidence and who consistently, and intentionally in some cases, failed us eventually steering us back on to the original track where we are today.
Since life moves in cycles we will always encounter those branches in the trail that present opportunity and require courage. Thus there can always be hope that we will either willing choose to trace an alternate cycle or accidentally stumble onto the less-worn path and decide that it is more interesting and then decide collectively that we should continue that adventure.
There are too many insightful and thoughtful points in your comment for a single upvote to repay, so I must thank you with a reply. Becoming so demoralized with the present media and political landscape, it is comforting to find that there still remain semblances of reason for which I may again briefly restore some measure of sanity. Thank you.
Thank you for reading all of that. I read a lot and tend to be long-winded when I write and I know that a lot of people's attention spans are pretty short. I'm glad you found something of value in the words. That was my hope.
Of course today, half the population would start thinking this was some vast conspiracy (the astronauts were just sent to a soundstage in arizona) and the other half would be tweeting "I don't care about 3 overprivileged men"
There were also protests of the Apollo launches in Florda. But in 2022, it wouldn't be actual oppressed people making a statement; it would be rich white college students!
Every few years I re-watch the Running Man - and it gets scarier every time. Last time I watched it we only nearly had the video forgery technology from the movie (in civilian hands, at least). Now it actually exists.
This is amazing, though I think the voice is one of the hardest problems for deepfakes. I find there is a lot of inflection and tonality missing from synthesized voices. Even with a 'old TV broadcast' sound filter to mask some of that stuff, I think people are quite good at recognizing when a voice isn't totally human :) But the visuals in this production are incredibly convincing to me.
Those capture inflection and tonality quite well, and are able to reproduce near-natural sounding laughter, pauses, and other human qualities. I wouldn't say it's perfect just yet, but give it another ~5 years and it will get there.
What's amazing to me is the possibility to experience the "What-if" scenario, for a second. I now have a memory of watching the moon landing fail, even though this never happened, it's still a real memory that I can attach thought and emotion to.
One of the problems with deep fakes is that they open the door for plausible deniability in case of hidden cam/mic recordings of wrongdoings of public figures. (Think celebrities, politicians.)
"The UK national electrical grid delivers power across the country. This mains power supply makes a constant humming sound, yet there are tiny changes to the frequency of this sound every second. Most recordings made in the UK have a trace of mains hum on them and this can be forensically analysed to determine the time and date they were made, and as a result, whether anyone has edited the recording."
I'm really confused by the intended narrative here. There is some kind of disaster on route to the moon, then suddenly Nixon's speech that Armstrong and Aldrin are stranded on the moon. We skipped the landing?
Waste of time. 4 minutes of useless filler, followed by 2 minutes of bad voice synthesis. Not convincing at all, and all the padded content just serves to get your hopes up.
I'm not american and I can't recall if I heard him before, but the voice definitely felt a bit robotic. Also, he seemed to move his head in unnatural ways (even if he has some sort of nervous tick). Finally, if you look at his neck, jaw and shadow movement, you'll see some weird clipping going on. Granted, if I was not looking for it, I may haven't noticed.
All of the comments on how easy this was to spot are laughable. Of course it was easy to spot. The moon landing DID in fact happen and we all know it. Let’s see you try again when you have no idea you could be looking at a deepfake.
The questions at the end don't ask whether you think the moon landing happened they ask whether you could tell if individual components of the video were faked or real.
