ChatGPT-Assisted University Email Addressing Mass Shooting Denounced by Students

Vanderbilt University's Office of Equity, Diversity and Inclusion used ChatGPT to write an email addressing student body about the 2023 Michigan State University shooting, which was condemned as "impersonal" and "lacking empathy".

Risk classification

• Primary risk domain: 5 Human-Computer Interaction
• Primary risk subdomain: 5.1 Overreliance and unsafe use

The incident represents an inappropriate reliance on an AI system by university administrators to generate a message requiring genuine human empathy and personal connection during a tragedy.

Additional risk subdomains

• 6.3 Economic and cultural devaluation of human effort: The automation of grief and empathy devalued the expected human effort and sincerity required in times of tragedy.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The incident arose because university staff intentionally chose to use a deployed AI system to draft a sensitive email, resulting in an unexpected public backlash.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The system used is ChatGPT, a chatbot. According to the EU AI Act definitions, chatbots and systems that generate text require transparency obligations to notify users that they are interacting with an AI or that the content is AI-generated.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Chatbot
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: Vanderbilt University
• Alleged harmed parties: Vanderbilt University students, Vanderbilt University

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Psychological

Reported: Yes, the reports describe students feeling disgusted, outraged, and insulted by the sterile and impersonal nature of the AI-generated condolence email.

Directly caused: The email caused emotional distress, anger, and a sense of betrayal among students, particularly those with personal connections to the Michigan State University shooting, who felt the administration lacked genuine empathy.

Indirectly caused: N/A

Inferred additional harm: It is likely that hundreds of other students in the Peabody community experienced similar feelings of disappointment, alienation, or distress upon receiving the automated email, though they were not individually interviewed.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 2000

Potential causes

Management

• Lack of Administrative Oversight: Higher-ups were completely unaware of the email before it was distributed.
• Focus on Public Relations: Prioritized quick face-saving communication over genuine human connection.

Technology

• Lack of Emotional Intelligence: AI lacks empathy and judgment needed for tragedy communications.
• Model Inaccuracies: AI generated factual errors like 'recent Michigan shootings' in plural.
• Generic Text Generation: AI produced sterile, platitudinous, and boilerplate text lacking nuance.

Data Inputs

• Lack of Specific Context: AI was not trained on real-time event details, leading to generic outputs.
• Outdated Training Data: Model lacked up-to-date knowledge of current events of the past two years.

Human Factors

• Poor Human Judgment: Deans decided to outsource a sensitive, sorrowful letter to a chatbot.
• Desire to Save Time: Administrators used AI to quickly generate text instead of self-reflecting.
• Lack of AI Literacy: Staff did not grasp the emotional inappropriateness of using AI for grief.

Process and Methods

• Bypassed Review Processes: The email did not go through the normal multiple layers of review.
• Inadequate AI Guidelines: No clear procedures or policies existed for using AI in official emails.
• Lack of Safety Checks: No verification was done to ensure the message met community standards.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are highly consistent, detailed, and come from multiple reputable news sources as well as the student newspaper. The facts of the incident—including the text of the email, the footnote, the reactions of the students, and the subsequent administrative actions—are clear and undisputed.

Vanderbilt University administrators used OpenAI's ChatGPT to draft a condolence email following a mass shooting, sparking student outrage over the perceived lack of empathy. The incident represents a localized public relations issue and organizational failure regarding AI overreliance, carrying negligible implications for national security.

• Overall national security impact: Negligible
• Response level: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Vanderbilt University administrators

Threat characteristics

• Imminence: Long-term. The incident is a resolved localized administrative issue with no ongoing or impending national security threat.
• Autonomy: Human-controlled. The AI was used as a writing assistant to draft text, with humans ultimately reviewing and executing the communication.
• Novelty: Evolved capability. Represents an evolution in the public perception and social risks of using generative AI for sensitive, empathy-driven human communications.

Impact by dimension

• Physical security: Negligible. The incident involves an administrative email at a university and has no impact on physical systems, critical infrastructure, or kinetic safety.
• Information security: Negligible. There is no evidence of foreign intelligence operations, systematic disinformation campaigns, or compromise of classified information.
• Sovereignty: Negligible. The incident is limited to a private university's administrative communications and does not affect state authority, federal government operations, or electoral systems.
• Economic security: Negligible. No strategic economic assets, financial systems, or competitive technological advantages were compromised or targeted in this incident.
• Societal stability: Negligible. Although the email caused localized distress and outrage among students, it posed no threat to national societal stability, civil liberties, or broader population safety.