Purportedly AI-Generated Facebook Pages Allegedly Targeted Australian Rules Football Club Geelong Cats With Fabricated Harmful Claims

Purportedly AI-generated Facebook pages reportedly published false stories about the Geelong Football Club, including fabricated criminal allegations, illnesses, deaths, and a nonexistent bushfire evacuation warning. The club reported the pages to Meta, which later removed at least one page after complaints, while other misleading content reportedly remained online.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves malicious actors using AI-generated content and fake pages to impersonate club figures and spread false stories, aiming to drive traffic to suspicious links for personal or financial gain.

Additional risk subdomains

• 3.1 False or misleading information: The AI-generated posts spread fabricated news stories, fake arrests, and medical diagnoses, creating epistemic harm for fans.
• 1.2 Exposure to toxic content: Some posts contained darker, exploitative content targeting players' personal lives and alleging criminal activity.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human actors who intentionally created fake Facebook pages and used AI tools to generate and spread false stories for engagement.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The incident involves 'AI-generated content such as deepfakes' and synthetic media, which under the EU AI Act are subject to transparency obligations to ensure users are aware they are interacting with AI-generated content.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Social Media Content Generation; Deepfake Image Generation
• Behaviour type: Tool
• Alleged developer: Synthetic media generation technology developers, Synthetic image generation technology developers, Deepfake technology developers
• Alleged deployer: Synthetic media creators, Information manipulation actors in Australia, Information manipulation actors
• Alleged harmed parties: Social media users, Residents of Geelong, Victoria, General public of Australia, General public, Geelong Football Club, Geelong Cats players, Geelong Cats fans, Geelong Cats, Facebook users, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report explicitly describes false and malicious AI-generated content, including fake arrests and exploitative posts targeting players' personal lives.

Directly caused: The fake pages generated and spread malicious posts, including fake arrest stories and fabricated tributes, to thousands of followers on Facebook.

Inferred additional harm: Additional toxic content may have been generated and shared across other social media platforms, exposing thousands of more users.

Privacy

Reported: Yes, the report notes that contact numbers belonging to real individuals were used on the fake pages without their knowledge.

Directly caused: The phone numbers of a Grovedale resident and a Bendigo man were published on the fake pages without their consent.

Inferred additional harm: The personal details of other individuals or players may have been scraped or misused to set up these accounts, though not detailed in the report.

Psychological

Reported: Yes, the report notes the content is concerning, particularly when content involves our people and describes exploitative posts targeting current and former players' personal lives.

Indirectly caused: The players and staff targeted by fake criminal allegations, medical diagnoses, and personal life exploitation likely experienced distress and anxiety.

Inferred additional harm: It is likely that targeted players, their families, and fans who fell for the scams experienced emotional distress, though specific numbers are not quantified.

Epistemic

Reported: Yes, the report describes the spread of AI-generated misinformation, fake news stories, and fabricated events presented as legitimate news.

Directly caused: The pages published fabricated stories about player trades, stadium redevelopments, and a fictional player, misleading fans who follow the pages.

Indirectly caused: The club had to issue warnings to fans to stick to official news sources, indicating an erosion of trust in online information about the team.

Inferred additional harm: The widespread dissemination of highly realistic fake news likely contributed to broader public confusion and a general erosion of trust in digital media.

People affected

• Occurrences reported: 1
• People reportedly harmed: 10
• People reportedly exposed: 2000

Potential causes

Management

• Platform Unresponsiveness: Meta failed to respond to requests for comment or take immediate action.

Technology

• Sophisticated AI Generation: AI tools enable creation of highly realistic fake images and news.
• Algorithmic Recommendations: Social media algorithms amplify fake pages to thousands of followers.
• Lack of Automated Detection: Platforms lack effective AI tools to automatically flag deepfakes.

Data Inputs

• Misappropriated Personal Data: Real player names, images, and contact details used to seed fake pages.
• Exploitation of Real Events: Using actual news events to make AI-generated stories appear credible.

Human Factors

• Susceptibility to Clickbait: Users are lured by sensational AI headlines to click dangerous links.
• Malicious Intent of Creators: Bad actors deliberately use AI to damage reputations and spread lies.

Process and Methods

• Inadequate Content Moderation: Meta's processes fail to quickly remove reported AI misinformation.
• Flawed Page Verification: Lack of robust identity checks allows fake news pages to grow unchecked.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the nature of the AI-generated content, the platform used (Facebook), the targets (Geelong Football Club and players), and the specific types of misinformation spread. While the exact AI models used to generate the text and images are not specified, the description of the incident's dynamics and harms is highly detailed and consistent.
• Ambiguities identified: The specific AI tools or models used to generate the text and images are not named.

A localized social media disinformation campaign targeted an Australian football club using AI-generated text and images. The incident has negligible national security implications, as it does not affect critical infrastructure, government functions, or national stability.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: Australia
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents a localized social media issue rather than a national security crisis requiring immediate government intervention.
• Autonomy: Human-controlled. AI tools were used by human operators to generate text and images, which were then manually posted to Facebook.
• Novelty: Established threat. The use of AI-generated content for online harassment and fake news on social media is a well-documented and established threat.

Impact by dimension

• Physical security: Negligible. No threat to physical systems, critical infrastructure, or human safety. The incident is confined to social media harassment.
• Information security: Negligible. No impact on national security intelligence, classified systems, or state-sponsored information operations targeting democratic institutions.
• Sovereignty: Negligible. No threat to state authority, electoral processes, or core government operations. The target is a domestic sports club.
• Economic security: Negligible. No threat to critical financial systems, strategic technologies, or national economic stability.
• Societal stability: Negligible. While targeted individuals experienced distress, there is no large-scale threat to social cohesion, civil liberties, or public safety.