Voice tool 'misused' as deepfakes flood web forum, AI company says

 

ElevenLabs tweeted its technology had been "overwhelmingly applied to positive use" but said "additional safeguards" were needed.

AI tools are trained on vast swathes of data – pictures, audio, text or video – that allow them to essentially predict what output the user would like. (Carol Yepes / Getty Images) 

 

A British AI firm's audio tool was utilised to clone celebrity voices and have them say racist and discriminatory slurs. ElevenLabs said it was rethinking its "safeguards" against such misuse.

ElevenLabs tweeted on Monday that it had been a "crazy weekend" and admitted to finding "an increasing number of voice cloning misuse cases" just days after it released a demo version of the tool.

The firm did not give details of the cases but the internet forum 4chan was flooded with deepfakes of celebrity voices using racist, sexist and discriminatory slurs.

 

 

 

Crazy weekend - thank you to everyone for trying out our Beta platform. While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to Twitter community for thoughts and feedback!

— ElevenLabs (@elevenlabsio) January 30, 2023

Many of the posts mentioned ElevenLabs, with some users claiming to have been banned by the firm and others providing links to the demo version of the tool.

Celebrities who came in for treatment on the forum, which is notorious for its offensive humour, included the stars of the Harry Potter films, and film directors Quentin Tarantino and George Lucas.

ElevenLabs has touted its voice cloning technology as having "the potential to revolutionise the way content is produced, delivered and interacted with across a range of industries".

In its publicity material, the firm produced a video of actor Leonardo DiCaprio being voiced by several other celebrities.

 

 

 

 

The firm tweeted that its technology had been "overwhelmingly applied to positive use" but said "additional safeguards" were needed.

Among the options it was considering were manually verifying each cloning request or tightening account verifications.

AI tools are trained on vast swathes of data – pictures, audio, text or video – that allow them to essentially predict what output the user would like.

Text tool ChatGPT has taken the world by storm with its ability to generate essays to a relatively high standard with just a few prompts.

Imaging tools like Dall-E 2 and Midjourney are threatening to upend the art and design world with their ability to generate complex images in the style of famous artists.

But critics have long expressed fear that cloning technology, particularly in audio and video, could lead to a flood of deepfakes and disinformation.

And several AI companies are facing lawsuits from people who allege the firms have breached copyright by using their material as a basis for the AI models.

Source: TRTWorld and agencies 

Social media is bold. 

Social media is young.

Social media raises questions.

 Social media is not satisfied with an answer.

Social media looks at the big picture.

 Social media is interested in every detail.

social media is curious.

 Social media is free.

Social media is irreplaceable.

But never irrelevant.

Social media is you.

(With input from news agency language)

 If you like this story, share it with a friend!   

We are a non-profit organization. Help us financially to keep our journalism free from government and corporate pressure .