Meta has identified several accounts which used AI-generated content on Facebook and Instagram to spread deceptive comments, such as praising Israel’s handling of the Gaza conflict, posted under articles from global news outlets and US lawmakers.
In its quarterly security report, Meta revealed that the accounts involved posed as Jewish students, African Americans, and other concerned citizens, targeting audiences in the United States and Canada.
The company attributed the campaign to STOIC, a political marketing firm based in Tel Aviv.
Meta has detected AI-generated profile photos in influence operations since 2019, but this report is the first to reveal the use of text-based generative AI, which emerged in late 2022,
Researchers worry that generative AI, capable of producing human-like text, images, and audio quickly and cheaply, could enable more effective disinformation campaigns and influence elections.
In a press call, Meta security executives stated they quickly shut down the Israeli campaign and did not believe new AI technologies hindered their efforts to disrupt coordinated influence networks.
They also noted they had not encountered AI-generated images of politicians that were realistic enough to be mistaken for genuine photos.
Influence networks are coordinated groups or systems designed to spread specific messages or narratives, often to sway public opinion, manipulate perceptions, or achieve certain political or social goals.
"There are several examples across these networks of how they use likely generative AI tooling to create content. Perhaps it gives them the ability to do that quicker or to do that with more volume. But it hasn’t really impacted our ability to detect them," said Meta head of threat investigations Mike Dvilyanski.
The report revealed that Meta disrupted six covert influence operations in the first quarter. Besides the STOIC network, Meta also shut down an Iran-based network focused on the Israel-Hamas conflict, although no generative AI was detected in that campaign.
Meta and other tech giants are grappling with how to prevent the misuse of new AI technologies, especially in elections.
Comments