AI-Generated Airline Reviews Allegedly Mislead Consumers and Undermine Trust

AI-generated reviews of airline services have reportedly increased by 189% since the release of ChatGPT, with certain carriers like China Southern Airlines and SouthWest Airlines disproportionately affected, according to a study by Originality.ai.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The generation of fake reviews constitutes a form of fraud and targeted manipulation of consumer beliefs for commercial or reputational advantage.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The massive influx of AI-generated fake reviews pollutes online review platforms, undermining the authenticity and reliability of the information ecosystem.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident is caused by human actors intentionally using deployed AI systems to generate and post fake reviews to manipulate online ratings and consumer perceptions.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves the use of AI systems to generate synthetic text content (fake reviews), which falls under transparency obligations for AI-generated content.

AI system and alleged parties

• AI system: ChatGPT, Originality.ai AI Detector (OpenAI, Originality.ai)
• AI purpose: Writing Assistant; Social Media Content Generation
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: Unknown reviewers
• Alleged harmed parties: Airlines, Airline customers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes malicious content in the form of fake, AI-generated reviews spread across online platforms.

Directly caused: A massive volume of fake reviews was generated and posted online, with China Southern Airlines alone having 32.4% of its analyzed reviews flagged as AI-generated in 2023.

Inferred additional harm: Widespread proliferation of fake reviews across other consumer sectors and platforms beyond the 10 analyzed airlines.

Epistemic

Reported: The report explicitly describes epistemic harm, noting that fake reviews spread misinformation and erode online authenticity and trust.

Directly caused: AI-generated reviews mislead consumers by fabricating customer experiences, blurring the line between authentic and artificial feedback.

Indirectly caused: A 90% average decline in customer sentiment and a general loss of consumer trust in the authenticity of online airline reviews.

Inferred additional harm: A broader erosion of shared truth and trust in all online user-generated feedback systems, making it difficult for consumers to make informed decisions.

People affected

• Occurrences reported: 1

Potential causes

Management

• Neglecting Review Authenticity: Airlines and platforms fail to actively manage and verify review integrity.
• Prioritizing Reputation Metrics: Focus on ratings over ensuring reviews reflect genuine customer experiences.

Technology

• Advanced LLM Text Generation: LLMs like ChatGPT generate highly realistic reviews mimicking human styles.
• Lack of Platform AI Detection: Review platforms lack built-in tools to detect and block AI-generated reviews.

Data Inputs

• AI-Generated Review Data: Reviews with repetitive structures or no personal details pollute platforms.
• Biased Sentiment Datasets: Fake extreme reviews skew the overall sentiment and rating data of airlines.

Human Factors

• Inability to Detect AI Content: Humans struggle to spot AI reviews, leading to trust in fake feedback.
• Ease of Prompting for Manipulation: Experienced prompters can easily generate multiple unique reviews in seconds.

Process and Methods

• Inadequate Review Verification: Platforms lack processes to confirm reviews are from real customers.
• Lack of Review Transparency: No clear disclosure of review authenticity or whether AI detection is used.

Regulatory Environment

• Lagging Anti-Fake Review Policies: Slow implementation of regulations to combat AI-generated fake reviews online.
• Lack of Global Standards: Disparities in regulations, though some regions are enacting policies.

Information quality

• Classification confidence: Medium
• Reason for confidence: The report provides detailed statistical trends on AI-generated reviews across major airlines. However, the study was conducted by a commercial AI-detection company (Originality.ai) using its own proprietary tool, which introduces potential bias and lacks independent peer-reviewed validation. Additionally, the exact actors generating the reviews are not identified.
• Ambiguities identified: It is unclear who is generating and posting the fake reviews (e.g., the airlines themselves, competitors, or third-party marketing agencies).
• Alternative interpretations: The decline in customer sentiment could be primarily driven by actual service quality issues (such as post-COVID operational challenges) rather than being directly caused by the rise of AI reviews.
• Missing information: Independent verification of the AI detector's accuracy on this dataset, and concrete evidence identifying the entities responsible for commissioning or posting the fake reviews.

The incident involves a significant increase in AI-generated fake airline reviews, eroding consumer trust in online platforms. From a national security perspective, the impact is negligible as it is confined to commercial consumer feedback and does not threaten critical infrastructure, national intelligence, sovereignty, or strategic economic systems.

• Overall national security impact: Negligible
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United States, China, United Kingdom, Canada, Qatar, United Arab Emirates, Ireland
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The proliferation of AI-generated reviews represents an ongoing strategic concern regarding online information authenticity rather than an immediate crisis.
• Autonomy: Human-controlled. The AI is used as a writing assistant to generate text based on specific human prompts, and humans are responsible for posting the content.
• Novelty: Evolved capability. Fake reviews are an established threat, but the use of large language models to generate realistic, unique-toned feedback at scale represents a significant evolution in capability.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety elements were compromised or targeted in this incident.
• Information security: Negligible. While the incident involves AI-generated synthetic text, it is limited to commercial airline reviews and does not compromise national intelligence, classified models, or state-sponsored information operations.
• Sovereignty: Negligible. The incident does not impact state authority, electoral systems, government decision-making, or core government operations.
• Economic security: Negligible. The generation of fake consumer reviews affects commercial trust but does not pose a threat to financial systems, strategic industries, or national technological competitive advantage.
• Societal stability: Negligible. There is no indication of threat to social cohesion, civil liberties, mass surveillance, or human rights.