Using an LLM pipeline, we assessed whether each document in CSET Emerging Technology Observatory's AGORA dataset included mention of any one of 10 technical terms that define the scope of AI systems addressed in governance documents. See the pilot blog post for discussion of the methodology.
Insights from analysis of Technical Scope terms mentioned:
- Coverage patterns: “AI Systems” and “AI Models” receive broad coverage, “Task Specific AI”, “Predictive AI”, and “Generative AI” receive substantial attention, however less than “AI Systems” and “AI Models”. Because AI Systems and AI Models encompass a wide range of technologies and use cases, this pattern suggests that current governance documents tend to regulate AI in general terms
Interactive Chart
Explore the chart using the dropdown filters, and by clicking on category names (Jurisdiction, Authority, Legislative Status etc) to see distributions within each category or click on the preset example configurations below the chart.
[{ "label": "Technical scope terms in enacted documents only, broken down by Jurisdiction", "filters": { "activity": ["Enacted"], "stackBy": "jurisdiction" } }, { "label": "Technical scope terms in US documents only, broken down by Authority", "filters": { "jurisdiction": ["United States"], "stackBy": "authority" } }
]
Important context for interpreting these results:
The LLM classifications are based on the approximately 1000 documents in the AGORA dataset, which is predominantly composed of U.S.-origin English language government proposed documents, the majority of which are federal-level.
Coverage patterns described therefore reflect the priorities and framing conventions of this particular corpus and should not be taken as representative of the global AI governance landscape. We also found LLM classifications to exhibit some biases including over-attribution of coverage when governance-related language is present. Coverage scores should be taken as indicative of broad patterns rather than precise measurements.
%20(250%20x%20100%20px).png)
.png){kind=small}
