Last week I published an AI video quiz that got picked up on Reddit, Hacker News, and X, and it received a decent amount of attention.

The quiz itself consists of 10 questions, each of which is a video of a notable person speaking, and asks the user to try to identify whether or not the video is real or AI generated.

When I first shared the quiz with my friends, a lot of the feedback that I got was that it was way to easy. Many Redditors agreed as well:

However, while many of the commenters might have said the quiz was easy, the actual data shows something else entirely. Out of the people who successfully finished the quiz (meaning they answered all 10 questions), only 6.7% of respondents were able to answer all 10 questions correctly.

Since this number didn't match up with what I was hearing anecdotally, I wanted to share this write-up about the results and some key takeaways that I had, especially in the context of the upcoming elections.

The Results

Warning: answers below. If you'd like to take the quiz first, feel free to try it here.

We threw analytics on the page a little late, but we managed to sample 545 total quiz takers. Out of this number, 394 actually completed the quiz (answered all 10 questions). From this sample, here are some of the initial results:

The most common score was 6/10, which 22% of quiz takers got.

Answering 10/10 questions correctly was the 7th most likely result, which only 6.7% of quiz takers scored.

The hardest video to get right was the video of Donald Trump shilling his coin. 60% of users thought it was a fake video, when in fact it was real.

The most realistic fake video was the video of Tom Cruise at the Producer's Guild Awards. 52% of respondents said it was a real video, when in fact it was AI-generated.

The second most realistic fake video was the video of Mark Zuckerberg at Harvard Commencement. It was split 50-50 almost exactly, with 50% of respondents thinking it was real and 50% thinking it was fake. The actual video is AI-generated.

Takeaways

The quiz was only meant for fun and to raise awareness for how far AI generation technologies, especially in the video space, have come. All of the videos were made with software that is currently available on Kapwing in the form of AI Personas. While we were building these features for marketers to use, we realized that they could be used in nefarious ways and wanted to make it known how powerful they can be in the hands of bad actors.

One clear takeaway from the results is that we may be overestimating our own ability to distinguish what's real from what's fake. Many users on Reddit and X claimed that the quiz was far too easy. It's possible that they individually scored well on the quiz, but the aggregate results go to show that the vast majority of people can't tell the difference between the real and fake videos. In this case, they were used for educational purposes, but if they were instead repurposed to other use cases, they could be dangerous.

Another takeaway is to realize how far individual technologies have come. The improvements in individual technologies like voice cloning, lip sync, and AI video editing have all caused these types of video to be possible. This technology will only improve going into the future, and is something that we need to be aware of as a society.