A tourism website, Tasmania Tours, published AI-generated content that invented a nonexistent attraction called 'Weldborough Hot Springs'. This led to tourists traveling to the remote town of Weldborough, causing significant confusion and disruption for local businesses. The company owner admitted the AI content was used to compete for search engine prominence and was published without proper review.
A purportedly AI-generated travel article on the Tasmania Tours website reportedly invented "Weldborough Hot Springs" and promoted them as a real attraction. Tourists allegedly traveled to the remote town of Weldborough in search of the site, while local businesses reportedly received repeated calls and visits from confused travelers asking how to find the nonexistent springs.
Risk classification
- Primary risk domain: 3 Misinformation
- Primary risk subdomain: 3.1 False or misleading information
The AI system generated false information about a non-existent hot spring, leading tourists to travel to the location based on inaccurate beliefs.
Additional risk subdomains
- 7.3 Lack of capability or robustness: The AI system exhibited a clear lack of capability by hallucinating non-existent landmarks and generating physically impossible images of animals.
Causal factors
- Entity: AI
- Intent: Unintentional
- Timing: Post-deployment
The incident was caused by an AI system generating hallucinated content post-deployment, which was unintentionally published without human review.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The system generated text and images ('AI-generated blog posts') that were published online, which falls under the category of AI-generated content requiring transparency so users know they are interacting with synthetic media.
AI system and alleged parties
- AI system: unspecified
- AI purpose: Writing Assistant; Image Generation
- Behaviour type: Assistant
- Alleged developer: Unknown image generator developers, Unknown generative AI developers
- Alleged deployer: Unknown third-party marketing provider(s), Tasmania Tours, Australian Tours and Cruises
- Alleged harmed parties: Weldborough Hotel, Weldborough businesses, Tourists seeking Weldborough Hot Springs, Kristy Probert, Epistemic integrity
Harm severity
Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Negligible, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Negligible, indirect Negligible
- Differential treatment: direct Negligible, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Negligible, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Negligible, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
People affected
- Occurrences reported: 1
- People reportedly harmed: 30
- People reportedly exposed: 50
Potential causes
Management
- Outsourcing Marketing Content: Company outsourced content creation to third parties using AI.
- Pressure to Compete: Need to keep content refreshed to compete with larger booking sites.
Technology
- AI Hallucination: AI model generated non-existent destinations like Weldborough Hot Springs.
- Flawed Image Generation: AI created unrealistic images of animals like three-legged wombats.
Human Factors
- User Trust in AI Itineraries: Travellers relied on AI-generated travel plans without verification.
- Accidental Public Release: Content was published by mistake while the owner was out of the country.
Process and Methods
- Inadequate Review Process: Outsourced AI content was published without passing the standard review.
- Lack of Quality Control: No robust mechanism to verify accuracy of AI-generated travel details.
Regulatory Environment
- Lack of AI Content Regulation: Absence of regulatory standards for verifying AI-generated tourism info.
Information quality
- Classification confidence: High
- Reason for confidence: The report provides clear, direct accounts from the business owner, local publican, and tourism experts. The role of the AI in generating the false content is explicitly confirmed by the business owner, leaving little ambiguity about the cause and effects of the incident.
- Ambiguities identified: The specific AI model used by the third-party marketing agency is not named.
- Alternative interpretations: None. The owner explicitly blamed the AI for completely messing up and generating the fake content.
A local Australian tourism website published unreviewed AI-generated articles fabricating a non-existent tourist attraction. While causing minor local confusion and travel detours, the incident represents a standard commercial AI hallucination with negligible national security implications.
- Overall national security impact: Negligible
- Response level: Minor
- Scope: Single nation
- Primary target: Australia
- Alleged perpetrator: Tasmania Tours
Threat characteristics
- Imminence: Long-term. The false content has been removed and the incident is resolved, representing no active crisis or immediate national security threat.
- Autonomy: Human-supervised. The AI system functioned as a content generation tool intended for human review, though it was published without oversight due to process failure.
- Novelty: Established threat. AI hallucinations and the accidental publication of unverified synthetic content are well-established and common issues.
Impact by dimension
- Physical security: Negligible. No physical infrastructure, military systems, or human safety was threatened. The incident only involved minor travel detours by tourists.
- Information security: Negligible. The false content was generated unintentionally for search engine optimization by a local business, with no connection to hostile disinformation or intelligence compromise.
- Sovereignty: Negligible. The incident was entirely commercial and localized, posing no threat to government operations, elections, or state authority.
- Economic security: Negligible. Economic impact was limited to minor fuel expenses for a small number of tourists and lost time for local businesses, presenting no strategic economic threat.
- Societal stability: Negligible. The incident caused minor localized confusion and did not impact social cohesion, human rights, or societal stability.