AI Work Assistants Require More Effort Than Expected, CIOs Say

AI work assistants, such as Copilot for Microsoft 365 and Gemini for Google Workspace, are proving to be more labor-intensive than anticipated for enterprises. CIOs report that these AI tools struggle with outdated or inaccurate data, leading to incorrect outputs. Companies are finding they must invest heavily in data management to ensure reliability. This added effort has led to delays in deployment and frustration, as businesses work to maximize the potential of these expensive AI tools.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI systems failed to perform reliably, delivering incorrect answers about executive teams, expense policies, and financial data due to technical immaturity and data retrieval limitations.

Additional risk subdomains

• 3.1 False or misleading information: The AI systems generated incorrect and contradictory information, leading to inaccurate beliefs or confusion among corporate users.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident involves AI assistants unintentionally generating incorrect or outdated answers during post-deployment corporate testing.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3. Limited Risk: The report describes chatbots and generative AI assistants (Copilot, Gemini) interacting with corporate users, which are subject to transparency obligations.

AI system and alleged parties

• AI system: Copilot for Microsoft 365, Gemini for Google Workspace
• AI purpose: Question Answering; Writing Assistant
• Behaviour type: Assistant
• Alleged developer: Microsoft, Google
• Alleged deployer: Enterprise teams, Companies in general, CIOs
• Alleged harmed parties: Microsoft Copilot users, Enterprise teams, Companies in general, CIOs

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly exposed: 1000

Potential causes

Management

• Premature Adoption Planning: Management attempts production deployment before data is ready.
• Underestimating Integration Effort: Leaders underestimate the internal lift and cost required for deployment.

Technology

• Immature AI Tooling: The generative AI tools themselves are still maturing and evolving.
• Lack of Source Prioritization: Tools cannot distinguish authoritative systems from emails or documents.
• Inadequate Context Retrieval: AI fails to ask users for more context when queries are ambiguous.

Data Inputs

• Outdated Enterprise Data: AI accesses stale data, leading to outdated or incorrect answers.
• Data Duplication and Inconsistency: Inconsistent and duplicated data causes contradictory AI outputs.
• Insecure Data Permissions: AI exposes sensitive historical files users should not access.

Human Factors

• Inadequate Prompting Skills: Users do not provide sufficient context in prompts to get accurate results.
• Unrealistic Expectations: Users expect magic without preparing necessary backend data infrastructure.

Process and Methods

• Poor Data Management Practices: Organizations lack processes to clean, validate, and update enterprise data.
• Lack of Authoritative Routing: No processes to direct AI to verified systems of record.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, consistent, and detailed journalistic accounts of corporate trials of AI assistants, naming specific companies, executives, and the exact nature of the AI failures.
• Ambiguities identified: None of significance.
• Alternative interpretations: None.

Major corporations testing generative AI work assistants reported retrieval errors and outdated outputs due to uncurated internal enterprise data. The incident represents standard commercial software adoption challenges with negligible national security implications.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident reflects ongoing software maturity and data management challenges rather than an active security crisis.
• Autonomy: Human-controlled. The AI systems function as corporate assistants where humans prompt the system and review the outputs, maintaining final decision-making control.
• Novelty: Established threat. Data quality issues, search retrieval failures, and AI hallucinations are well-established technical limitations of generative AI models.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety elements were impacted or threatened by the incorrect AI outputs during corporate testing.
• Information security: Negligible. No intelligence compromise, classified data theft, or systematic disinformation operations occurred; the issue was limited to internal corporate data retrieval.
• Sovereignty: Negligible. The incident involves commercial software testing within private corporations and has no impact on state authority, border control, or government operations.
• Economic security: Minor. Minor impact limited to corporate trial inefficiencies, wasted subscription fees, and increased labor costs for data curation at participating firms.
• Societal stability: Negligible. No threats to societal stability, civil liberties, or human rights were reported during these corporate AI assistant trials.