Autonomous Kargu-2 Drone Allegedly Remotely Used to Hunt down Libyan Soldiers

In Libya, a Turkish-made Kargu-2 aerial drone powered by a computer vision model was allegedly used remotely by forces backed by the Tripoli-based government to track down and attack enemies as they were running from rocket attacks.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.2 Cyberattacks, weapon development or use, and mass harm

The incident involves the wartime deployment of a lethal autonomous weapon system (LAWS) designed to hunt and engage human targets, representing the use of AI-enhanced weaponry.

Additional risk subdomains

• 5.2 Loss of human agency and autonomy: The deployment of 'fire, forget and find' systems removes human oversight and decision authority from life-and-death targeting decisions.

Causal factors

• Entity: AI
• Intent: Intentional
• Timing: Post-deployment

The incident involved the active deployment of an autonomous weapon system programmed to intentionally target and engage retreating forces during a military conflict.

EU AI Act risk tier

• Risk tier: 1 Unacceptable

Unacceptable Risk: The system is described as a lethal autonomous weapon system ('killer robot') capable of targeting and killing humans without human intervention, posing an extreme danger to society and violating fundamental human rights.

AI system and alleged parties

• AI system: Kargu-2 (STM)
• AI purpose: Autonomous Drones; Target Identification
• Behaviour type: Autonomous
• Alleged developer: STM
• Alleged deployer: Tripoli-based government
• Alleged harmed parties: Libyan soldiers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Civil rights

Reported: Yes, the reports describe concerns from human rights organizations regarding the erosion of human control and judgment in life-and-death decisions.

Directly caused: The deployment of fully autonomous lethal weapons to target humans represents a direct challenge to the fundamental right to life and meaningful human control over lethal force.

People affected

• Occurrences reported: 1
• People reportedly exposed: 50

Potential causes

Management

• Heavy Military Investment: Militaries globally are investing heavily in autonomous weapons systems.
• Deployment of Advanced AWS: Decision by military forces to deploy Kargu-2 drones in active combat.

Technology

• Brittleness of AI Systems: AI systems can easily make mistakes like misidentifying civilian targets.
• Fire Forget and Find Programming: Drones were programmed to attack without requiring data connectivity.

Data Inputs

• Potential Algorithmic Bias: Algorithmic bias in AI systems can lead to fatal misidentifications.
• No Operator Data Connection: The drone relied entirely on local sensor inputs without human override.

Human Factors

• Erosion of Human Judgment: No human operator was in the loop to make life-and-death decisions.

Process and Methods

• Autonomous Target Selection: Targeting procedures allowed the drone to choose targets autonomously.

Regulatory Environment

• Lack of International Treaty: No international ban exists to prevent the deployment of autonomous weapons.
• Failure to Regulate AWS: The race to regulate autonomous weapons is being lost globally.

Information quality

• Classification confidence: Medium
• Reason for confidence: The reports are based on a UN Panel of Experts report, which is highly credible, but the UN report itself is cautious and does not confirm whether the drone was operating autonomously at the exact time of the attack, nor does it confirm any casualties.
• Ambiguities identified: It remains unconfirmed whether the drone was operating in autonomous or manual mode during the specific attack, and whether any casualties actually occurred as a result of the drone's actions.
• Alternative interpretations: The drone may have been operated manually by a human operator during the engagement, meaning it did not act as an autonomous 'killer robot' in this instance.
• Missing information: Definitive confirmation of whether the drone was in autonomous mode during the attack, and verified data on whether any casualties or injuries were caused by the drone.

First reported case of an AI-powered autonomous weapon system identifying and engaging human targets without human intervention in armed conflict. The deployment of the Kargu-2 drone in Libya establishes a concerning precedent for lethal autonomous weapons in warfare, with substantial international security and human rights implications.

• Overall national security impact: Severe
• Scope: Single nation
• Primary target: Libya
• Alleged perpetrator: Turkey

Threat characteristics

• Imminence: Long-term. Represents ongoing capability deployment and strategic concern rather than an immediate crisis requiring urgent response.
• Autonomy: Full autonomy. AI system made lethal targeting decisions and engaged targets without human intervention or real-time data connectivity.
• Novelty: First-of-its-kind. First reported case of a lethal autonomous weapons system potentially hunting and engaging human targets in an armed conflict.

Impact by dimension

• Physical security: Severe. Autonomous weapon system (Kargu-2 drone) hunted and engaged human targets without intervention, representing significant kinetic threat capability.
• Information security: Negligible. No significant intelligence compromise or information warfare operations indicated in the incident reports.
• Sovereignty: Substantial. Foreign military technology deployed against forces within Libya's sovereign territory during internal conflict over governmental control.
• Economic security: Minor. Tactical disruption of military logistics convoys, though systemic economic or technological security impacts were limited.
• Societal stability: Substantial. Deployment of autonomous lethal weapons targeting humans represents a direct challenge to human rights and meaningful human control over lethal force.