LipSync videos with Custom Voices

Generating lipsync’d videos is amazing and they are even better with a custom voice. Companies like ElevenLabs enable you to create custom voices by uploading voice samples and we are very happy to announce that you can now use your custom ElevenLabs voice inside of the Gooey.AI Lipsync Maker (and Copilot) Workflows.

Step 0 - Sign up on Eleven Labs

Sign up on Eleven Labs: https://elevenlabs.io/sign-up

Step 1 - Add a custom voice in Eleven Labs

Go to the Voice Lab section (https://elevenlabs.io/voice-lab) and click on “Add Generative or Cloned Voice”

You will be shown multiple options for where to generate the new voice from. As of now, they generated voices and community voices for free users. They offer Instant Voice Cloning and Professional Voice Cloning for their subscribers.

Generated voices are created by adjusting parameters such as gender, age, and accent. This doesn’t take audio samples, and so it might not be exactly like you had imagined.

Community voices are those made publicly available by other users. Eleven Labs does have a good collection of community voices but you might not find exactly what you are looking for.

If you subscribe for their lowest paying tier that costs $5 / month, you can make use of Instant Voice Cloning to extract a voice from audio samples within seconds. We’ll use this to create a clone of Nelson Mandela’s voice, so we can recreate his speeches from text.

For our use case, we converted two of his interview clips – Nelson Mandela’s special message to INTERPOL 75th General Assembly and “Nelson Mandela’s message to BFS” – to mp3 format, and uploaded them in the Eleven Labs instant voice cloning interface.

We agree to the

Note: there is more to know about the kinds of custom voices and you can visit their VoiceLab documentation to know more about it.

Step 2 - Copy API key from Eleven Labs

On the Eleven Labs website, click on the profile icon in the top right corner, and select the Profile option.

This will open up the Profile Settings and show you the API key. Make it visible and copy it to use in Gooey.

Source for these images is the Eleven Labs documentation.

Step 3 - Use it on Gooey’s Lip Sync with Text recipe

Visit the workflow page for LipSync with Text and add a video or image you want to extract the face from in “Input Face”. You will need to remove the default uploaded file and upload your own video or image file.

Open the “Settings” dropdown, and in “Voice Settings”, set the “Speech Provider” to “Eleven Labs”.

Select the checkbox that says “Use custom API key + Voice ID” and enter the copied API key into the field that says “Your Eleven Labs API Key”

This should fetch all the available voices from your account into the “Voice ID” field.

Choose the voice that you want to run in the Voice ID field. In our case it is the voice we added in Eleven Labs with the name “Nelson Mandela”, we select that.

Now we can copy some text from his 1964 speech, “I am Prepared to Die”, and add it into the “Input Text” field.

We hit “Submit” and voila! We get an accurate sounding version of the speech with a newer photo.

This is the run URL if you want to tweak and explore more: https://gooey.ai/lipsync-maker/?run_id=sbr6yi8i&uid=MPhrEpmVYkept8yJjsBzJPL0Tuj1

There are more Eleven Labs options that you can explore in the Gooey dashboard, such as the model to use and the voice settings. You can play around with them on Gooey and learn more about how to use them effectively from the Eleven Labs documentation.