Reportedly Faulty AI Transcription Threatens Integrity of Genoa Bribery Probe

A reported AI transcription software error in a Genoa bribery investigation is alleged to have incorrectly recorded "illicit financing" instead of "licit financing," which could have significantly impacted the case. The mistake was discovered during a review and is an example of the risks of relying on AI in judicial settings.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The speech-to-text software failed to perform robustly and accurately under real-world conditions, misinterpreting 'licit' as 'illicit' due to probabilistic bias.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The judicial system passively delegated transcription tasks to the software without establishing mandatory human review or verification protocols.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by a transcription error generated by a deployed speech-to-text AI system, which was an unexpected and unintentional outcome of its operation.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI system is used in the administration of justice (transcribing legal interrogations), which has significant implications for fundamental rights and public interests.

AI system and alleged parties

• AI system: speech-to-text software
• AI purpose: Voice Recognition; Judicial Decision Support
• Behaviour type: Tool
• Alleged developer: Unnamed automated transcription software developer
• Alleged deployer: Judiciary of Italy
• Alleged harmed parties: Roberto Spinelli, Paolo Emilio Signorini, Italian general public, Giovanni Toti, Genoa Prosecutor’s Office

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The report explicitly describes epistemic harm through the generation of a false transcript that led to misleading newspaper headlines.

Directly caused: The AI system generated a false transcription, replacing 'licit' with 'illicit' in an official legal document.

Indirectly caused: The false transcript was picked up by multiple newspapers, which published misleading headlines and spread incorrect information to the public.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Budget and Staff Cuts: Staff reductions forced rapid adoption of AI tools to save costs.
• Inadequate Risk Assessment: Management adopted AI tools without evaluating potential judicial risks.

Technology

• Probabilistic Word Prediction: Software predicts words based on probability rather than exact audio.
• Acoustic Confusion: Difficulty distinguishing similar sounding words like leciti and illeciti.

Data Inputs

• Biased Training Data: Training data from TV and news overrepresents illegal contexts.
• Lack of Contextual Training: Dataset lacks specific judicial context needed for accuracy.

Human Factors

• Passive Automation Bias: Users trust the software output blindly without critical verification.
• Inattentive Clerking: The court clerk failed to summarize or note the statement manually.

Process and Methods

• Lack of Human Verification: No mandatory human review process to verify automated transcriptions.
• Flawed Procurement Process: Software purchased like standard IT equipment without safety evaluations.

Regulatory Environment

• Absence of AI Regulations: No legal framework or standards govern the use of AI in courtrooms.
• No Professional Protocols: Lack of official guidelines and certified registries for transcribers.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed factual accounts of the transcription error, its consequences, and the underlying technical cause (probabilistic bias). The legal and systemic context in Italy is also well-explained.
• Ambiguities identified: The specific name and developer of the speech-to-text software are not identified.
• Alternative interpretations: None, the incident is clearly a technical failure of an AI transcription tool used in a judicial setting.

An automated speech-to-text tool used in an Italian court incorrectly transcribed 'licit financing' as 'illicit financing' due to probabilistic bias. The error compromised a judicial inquiry and led to false media reports before being corrected by defense lawyers. The incident highlights minor risks to sovereignty and civil rights through the unregulated adoption of AI in government functions.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Italy
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents an ongoing strategic concern regarding the integration of unverified AI tools in judicial systems rather than an active crisis.
• Autonomy: Human-controlled. The AI tool acts as an assistant to transcribe speech, while final legal decisions and verification remain under human control.
• Novelty: Established threat. Speech-to-text translation errors and probabilistic biases in AI models are well-documented and established technical limitations.

Impact by dimension

• Physical security: Negligible. The incident involved a software transcription error in a legal case, with no threats to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. No intelligence compromise or deliberate information warfare operations occurred. The false reporting was due to an unintentional transcription error.
• Sovereignty: Minor. The transcription error temporarily compromised a judicial inquiry, which is a core government function, but the error was caught and managed within standard legal procedures.
• Economic security: Negligible. No strategic technology theft, financial system attacks, or significant economic impacts were associated with this incident.
• Societal stability: Minor. The error directly threatened the suspect's civil rights and right to a fair trial, but the impact was localized to the individuals involved.