New AI video tools are fuelling violent racism on TikTok

Content warning: This story contains references to violence and racism.

TikTok creators are using racist AI-generated videos to amass followers and even make money through the platform, the Bureau of Investigative Journalism (TBIJ) can reveal.

We found a number of AI-generated videos – including one of small boats being threatened with a rocket launcher – that had racked up almost 8.5 million views despite breaching TikTok’s policies on violence and hate speech. It raises questions about the ability of its content moderation to detect and remove increasing volumes of harmful AI-generated content.

One clip with almost half a million views shows soldiers on a beach shooting out to sea while a voice shouts: “Fire at will! Target that dinghy!” The same user ran a TikTok Shop selling iPhone accessories and walkie talkies and would have received a cut, usually up to 20%, of each sale.

It is one of 11 accounts we found with active TikTok Shop listings that posted racist AI-generated videos in the past five months. Others show migrants being targeted with explosives or celebrate small boats being capsized. One shows an inflatable dinghy being blown up in a soldier’s crosshairs to the soundtrack of Another One Bites the Dust, and three included realistic depictions of news anchors perpetuating racism.

TikTok removed 23 videos in total after we contacted the company, and said that three of the accounts in question had generated sales through their TikTok Shop listings.

“Recent advances in AI, which show so much promise for good, are being subverted by people with extremist agendas – or in some cases just to make extra money,” said Andrew Rogoyski, a director at the Surrey Institute for People-Centred AI.

This wave of viral content comes after the release of new generative AI models by Google and OpenAI, which allow users to create videos through written instructions. They can now include granular detail and character voices.

Five videos reviewed by TBIJ appear to show a watermark indicating they were made with Google’s video generator Veo, whose latest version launched in the UK in May and rolled out over the summer, though in most cases it was unclear which tool had been used.

A Google spokesperson said the company had clear policies around the use of its AI products and the tools would continually get better at reflecting these policies.

A TikTok spokesperson said: “We have permanently banned multiple accounts flagged by this investigation and removed content which violated our rules”, adding that 99% of “violative content” on TikTok is removed proactively before a user reports it.

‘A cascade effect’

The accounts we found are seeking to exploit the rising anti-immigrant sentiment in the UK. A poll last month by Ipsos found that concern about immigration was at its highest level in a decade, while Reform UK, a party that recently announced a series of hardline immigration policies, has consistently led Westminster polls for several months.

A number of the videos we found refer to migrants as “invaders”. The most popular one, with nearly 4 million views, shows a group of South Asian people approaching the shore on a small boat as a white man shouts through a megaphone: “You can only come in if you promise to wear deodorant.” The creator’s TikTok Shop sold Monster energy drinks, creatine and lining papers for an air fryer.

“The design of [social media] platforms is to make profits out of engagement,” said Beatriz Lopes Buarque, a fellow at the London School of Economics working on online misinformation and the far right. “There is a huge tension between the economic incentives and the social and political impacts of this content, because hate is profitable.

“If it captures our attention, in the case of TikTok, we may watch the video until the end, and this is already enough for the algorithm to start recommending more of that [content], so we have a cascade effect.”

Rogoyski said: “We’re still in a ‘wild west’ situation where essentially anything goes, both with social media and now with the rise of generative AI.

“Nobody is being held accountable – the platforms that serve mis- and dis-information have abdicated responsibility for the problems that harmful content causes, whether teenage suicide, mental health problems or the rise of extremism.”

Last month, OpenAI launched Sora 2, the latest iteration of its text-to-video product, which allows users to add sound effects and dialogue to videos. While OpenAI says it prohibits users from making content that promotes violence, the Guardian found that, within hours of Sora’s launch, it had been used to create numerous videos featuring graphic violence and racism (though TBIJ did not identify any racist videos produced with Sora).

A spokesperson for OpenAI said the company had “built multiple layers of protection to prevent misuse” of Sora and added that the app was not yet available in the UK.

Monetising hate

The products for sale in TikTok Shop accounts reviewed by TBIJ were mostly cheap consumer goods. They included multipacks of instant ramen, cleaning products and electric massage guns as well as novelty items like a “remote control poop toy” and a “colour changing LED light bubble fish mood lamp”.

However, TBIJ separately found TikTok Shops selling merchandise with anti-immigrant slogans such as “Stop importing, start deporting” and “If this flag offends you I’ll help you pack”. One retailer of T-shirts and hoodies emblazoned with this slogan had sold more than 200 items for a total of almost £3,500.

Other accounts sold merchandise relating to the Unite the Kingdom march in September, which was Britain’s largest ever far-right protest according to the anti-extremist organisation Hope not Hate.

TikTok Shop is an e-commerce product which allows its users to sell products directly to their audience on the platform. More than £25bn worth of products were sold through TikTok Shop in 2024, according to market research platform Momentum Asia.

TikTok reported its UK sales were up 180% year-on-year at the end of 2024 and earlier this year Grocery Gazette reported that it had become one of the UK’s biggest sellers of toilet roll.

Greg Zakowicz, e-commerce advisor, said social media companies had been trying for years to expand into online retail and create “a singular platform for […] processing payments, for social media, for shopping”.

“The problem of social platforms or large marketplaces in general is the moderation. It’s a beast and you can’t capture everything, and… my guess is they would rather have something that slips through their cracks from a moderation standpoint, than wrongly moderating people out that will restrict sales.”

TikTok’s affiliate programme allows creators to host products sold by others and receive a commission for each purchase.

TikTok says affiliate sellers must follow its community guidelines, but our reporting indicates sellers posting harmful content can get around these checks.

Some accounts used other monetisation strategies. These include payments from the video editing platform CapCut (also owned by TikTok parent company Bytedance) for popular templates and selling online courses helping other users create viral content.