Today, we’re excited to launch AI Personas, a new way to create a video of yourself without having to record it.

Kapwing’s mission is to help video creators and creative teams leverage video as a storytelling medium. Since 2017, we’ve pitched the importance of video editing being as easy as drafting a document. This week, we added the latest advancement towards simplified video storytelling.

AI Personas enables creators to convert any text - a script, announcement, lesson, sales pitch - into a video without needing to record footage. From a 15 second training video, you can generate video footage of yourself or a stock AI Persona reading off any script you want. 

What is AI Personas?

Personas is a new feature in Kapwing that enables you to make an AI “twin” of yourself and use it to produce a video saying any script aloud. 

Once you’ve created an AI Persona from a 15-second training video and a one minute voice recording, you can repurpose it for any text blurb, script, or message. Create realistic video footage with your lips synced to the words. 

Creators can use their own face to make a customer persona or leverage one of the stock personas in our diverse library to read your script aloud.

Benefits of AI Personas

The upside of these tools is enormous when used ethically. For creators, the ability to generate video from a script or create virtual avatars can save a ton of time and open new doors for storytelling. It’s democratizing video creation by making it faster, more affordable, and accessible to more people. Plus, it could revolutionize industries like education, entertainment, and even therapy.

This feature significantly reduces the amount of time it takes to create footage. Most video experts know that a lot of time is spent bulk recording, re-recording, and editing out sections where you went off script. Personas stay on script with visual consistency through the whole video and can be updated like a word doc. Here’s some use cases we envision:

• Creators can fill in their videos with sections that they forgot to record. “Overdub” your existing footage with a matching Persona.
• Customer success and sales professionals will be able to create personalized videos at scale, going quickly from a pitch to a spoken video.
• People with speech impediments, vocal disabilities, or heavy accents will be able to tell their story without the criticism or judgment of others by replacing their voice with a generic voice.
• Communications teams will be able to keep videos up-to-date more easily with the ability to refresh and revise the script quickly, retroactively. 
• Lecturers will be able to make videos in multiple languages without re-creating video footage (We also use lip sync tech in our Dubbing suite)
• Thought-leaders can more quickly share their perspectives and insights, especially as a reaction to a timely story

Since quietly launching it to our existing customers last week, we’ve already seen small business owners putting it to use. We’re looking forward to updating AI Personas as we roll it out to more users and languages. 

Is this different?

Although there are other AI Avatar products online, Kapwing’s AI Personas are different in a few important ways. Rather than generating a character or animation from scratch, an AI Persona is trained on a real video clip and voice recording such that it can emulate you. This enables brands to ensure visual consistency across video clips. 

AI Personas are designed to be collaborative, meaning that you can share an AI Persona with colleagues or with a client. Within the Kapwing Brand Kit, authorized users can generate videos from one AI Persona for more brand consistency. Kapwing also does not force creators to take a real-time photo proving that their face matches the training video. Kapwing users must have permission to use the training video, but we think a mandatory check blocks legitimate use cases for teams, freelancers, actors, and consumers who give permission for AI generation outside of the platform.

You also do not need high-definition footage to train the AI Persona. Unlike other providers which need extensive training data, Kapwing’s AI Personas require just a few seconds of video and audio. The AI Persona is created instantly, while you wait. As a result, creators can experiment with different Persona videos easily, making several versions for different contexts. 

Personas is built on Kapwing’s powerful video editor. We believe that the most powerful use cases are when video footage is combined with other media layers, like Broll, background music, and overlays. 

How AI Personas work

Personas leverages new Lip Sync technology, which is available open source and by supported vendors. With Voice Clones powered by ElevenLabs, Personas creates an audio track that matches your voice, then syncs the lips of the video speaker to pronounce the words, making it look like a talking head video. 

Video edited on Kapwing

We built the Personas feature in partnership with engineering teams at Texels, Sync Labs, and Sieve, three lip sync API providers who specialize in the underlying generative models. Lip Sync vendors make it easier to generate realistic footage to match an audio track. Within Kapwing, video layers are synced on export, and users can choose which vendor works best for their face shape and structure. These three startups are proof of the face pace of development in the Lip Sync space. 

Although the quality of many Personas is not good enough to truly “fool” viewers, we think that it will get there over time and that the right training video and voice clone can lead to a highly-convincing Persona. Plus, we’ve also seen first-hand how quickly the lip sync technology is improving. You can try out sync's newest model for the first time today within Kapwing. 

In the meantime, AI Persona are valuable and believable as one layer within a longer video. We plan to add it into our AI Video Generator soon.

What about Deepfakes?

Personas is essentially a product that lets you deepfake yourself, and it also makes it easier to deepfake another person. If you can create lifelike footage of someone saying or doing something they never actually did, there’s room for manipulation, from fake news to personal blackmail. Another risk is that videos of creators will be used as AI Personas without their permission, even if only for benign or legitimate use cases. 

At Kapwing, we have an opportunity to help creators and set boundaries in the wild west of AI video creation. We think it’s important for well-intentioned companies to take on the burden of ensuring that rules exist, that there are structures and terms in place to deploy the technology ethically, and that users can be held accountable to their use and misuse. We’re a US-based company backed by top VCs and a solid engineering team, so I think we’re a better steward of this technology than the unregulated open-source web.

Here's what we've done to prioritize safety as part of the AI Personas launch:

Respecting rights of the creators

All Personas are created from video footage in the creative commons or footage recorded by our team for this specific purpose. We proactively notified all creators whose likeness may be used and established a takedown policy if an AI Persona is created of someone’s face without their permission. 

On Kapwing, users must have permission to use a video to train an AI Persona or a voice clone. In both creation flows, users must go through a “speed bump” to acknowledge that they have explicit permission from the person featured in the training video. It’s against Kapwing’s terms of service to make a Persona of someone without their permission, and we ban users who are found to violate this clause. 

Preventing abuse

People must be Kapwing customers to create a custom persona. This ensures that if abuse does occur, we will have visibility into who it comes from and can alert the authorities.

We developed monitoring tools that give us visibility into what personas are created and trace videos back to the author. We recently updated the Kapwing Terms of Service to cover illegal use cases for generative technology and enhanced our watermark for AI-generated content. 

Education

We've created educational materials like this blog post about how to tell the difference between AI generated and real video footage. In addition to two blog posts, there is a Help Center article about Personas with transparent information about our approach.

How Do I Try It?

Come to Kapwing, sign in, and click on the “Personas” tab to try out our AI Personas. You can try our stock personas for free or upgrade to Kapwing for Business for 30 minutes of Persona exports. Mobile support for Personas is still under development, so you’ll need to use a computer to try it today. 

Share your Persona video or tell us what you think about this feature us on X @KapwingApp!

Today, AI Personas is live on Product Hunt– give us some support there to help other creators discover Personas.