Interactive Artificial Intelligence Observatory
Faster Than Oversight
We may be building the most consequential technology in human history.
We are doing it as fast as possible.
Here is what we know.
Long before the first neural network was trained, long before silicon, before electricity, before the scientific method itself — human cultures were already telling stories about the dangers of artificial creation, ungovernable power, and knowledge pursued without wisdom. These fifteen myths, spanning four millennia and a dozen civilizations, are not predictions. They are pattern recognitions — deep intuitions about what happens when beings create forces they cannot control. Every one of them is now, in some precise and measurable way, coming true.
Something remarkable is happening. For most of human history, the boundary between machine and mind was clear: machines did what they were told, nothing more. That boundary is dissolving — faster than almost anyone predicted.
The Race No One Agreed To Run
The Artificial Analysis Intelligence Index measures AI model capability across reasoning, knowledge, and instruction-following. The frontier has advanced from single-digit scores to over 57 in just three years.
Each line tracks a lab's most capable model over time. The steepening curves show AI capability growing faster each year — and the gap between leaders shrinking.
Data: Artificial Analysis Intelligence Index — Last updated:
The Intelligence Index is a composite benchmark measuring model performance across reasoning, knowledge, mathematics, and coding tasks. Higher scores indicate greater capability. Used with attribution for educational purposes.
Raw scores are one thing. But the real question is: what can AI do on its own? Not answer a question. Not write a paragraph. Work — autonomously, for hours, across complex tasks. That horizon is doubling every four months.
Autonomy Horizon
How long a task takes a skilled human before AI agents complete it with 50% success. In 2019, AI could manage a 2-second task. By early 2026, that horizon had reached 12 hours of skilled work. It is doubling every 129 days.
This measures how long AI agents can work autonomously on real software engineering tasks. The y-axis is logarithmic: each gridline represents a tenfold increase in autonomous work duration.
Data: METR — Horizon Benchmark v1.1. Cite: arXiv:2503.14499 (NeurIPS 2025). Used with attribution for educational purposes.
The p50 horizon length is the task duration (for a skilled human) at which AI agents succeed 50% of the time. Shaded bands show 95% confidence intervals. The frontier line connects state-of-the-art models at time of release. The dashed trend line shows the 129-day doubling rate from 2023 onward.
The horizon is not just rising — it is multiplying. AI agents — systems that take autonomous actions in the world — went from two releases per quarter to nineteen in eighteen months. The tools are proliferating faster than the frameworks to govern them.
The Horizon Extends
AI agent releases per quarter have grown from 2 to over 40 in two years. New domains — customer service, healthcare, legal — are expanding rapidly alongside software development and computer use. The tools are proliferating faster than the frameworks to govern them.
Each bar shows new AI agent products released per quarter. The donut chart breaks down which industries these agents are designed for. Counts from 2025 onward are estimates.
Data: International AI Safety Report 2026, Figure 2.11 (67 agents, Dec 2024). 2025 quarterly counts estimated from public agent launch tracking.
Scores on a benchmark are one thing. But benchmarks are supposed to be hard — designed to test the limits of what machines can do. What happens when the limits keep moving? When problems designed to take a decade are solved in six months?
Closing the Gap
Seven benchmarks designed to measure the frontier of AI capability. Each line tracks the best model score over time. FrontierMath went from <2% to ~25%. ARC-AGI from ~5% to ~88%. SWE-bench from 2% to ~70%. Humanity's Last Exam — scored just 3% at launch — has already been pushed to 27% in months. Problems designed to resist progress for years are falling in months.
Each line tracks AI performance on a different standardized test. When a line approaches the top of the chart, AI has matched or surpassed human expert performance on that task.
Data: Epoch AI Benchmark Hub (CC BY 4.0). Individual sources: SWE-bench, Epoch FrontierMath, ARC Prize, GPQA, SimpleQA, MATH, Humanity's Last Exam.
Capabilities Index (ECI) is Epoch AI's aggregate score combining performance across multiple benchmarks into a single comparable metric per model.
Behind every model is a number: the raw computation used to train it — a proxy for ambition. It has been growing at five times per year since 2020. There is no sign it is slowing.
The Engine Room
Training compute for frontier AI models has grown by over 10,000× since 2020 — doubling every 9 months. Each point on this chart represents a decision by an organization to invest more computational resource than any previous model in history.
Each dot represents an AI model. The vertical axis shows the computing power used to train it — note the logarithmic scale, where each step represents a 10× increase. Training compute has grown roughly 4× per year since 2010.
Data compiled from Epoch AI research, published papers, and industry reports. Training compute estimates carry uncertainty — see Epoch AI documentation for methodology. CC BY 4.0.
Hardware Scaling: transistor counts from Our World in Data (Moore's Law), GPU FLOPS from manufacturer specifications (NVIDIA, AMD). Moore's original observation: Gordon Moore, "Cramming More Components onto Integrated Circuits," Electronics, 1965.
Training compute is only one axis of acceleration. AI development has entered a self-reinforcing feedback loop — a cycle in which AI systems are themselves accelerating the development of more powerful AI systems. Each stage operates on a shorter timescale than the last.
The Feedback Loop
AI development has entered a unique technological feedback loop: AI systems are now helping to develop new and more powerful AI systems. We can think of this as five stages, each with a shorter timescale than the last. Earlier stages are well underway. Later stages remain speculative — but could proceed very rapidly once unlocked.
Each stage feeds the next: infrastructure investment enables model development, models generate training data, data improves tools, tools enable networks, and networks could eventually lead to recursive self-improvement. The timescales compress at each stage — from years to potentially hours. Several stages are already underway simultaneously.
Framework: Aguirre, A. (2025). "Keep the Future Human: AI Safety Standards," Table 2: The AI Self-Improvement Cycle. Future of Life Institute.
This acceleration does not happen in a vacuum. It is driven by specific organizations, with specific capital, and a specific economic logic: the company that falls behind falls behind forever. Five companies now control most of the infrastructure, most of the talent, and most of the compute. Their incentive is speed.
Five Companies, One Direction
Apple, Microsoft, Alphabet, Amazon, and Meta now command a combined market capitalization exceeding $11 trillion — more than the GDP of every country except the US and China. These five companies are also the primary funders and deployers of frontier AI.
Stacked market capitalization of the five largest AI companies. The combined value now exceeds the GDP of most nations — a concentration of economic power unprecedented in the technology sector.
Data: Yahoo Finance — Monthly closing prices × shares outstanding — Last updated:
Behind those five companies are five more — the ones who manufacture the silicon that makes AI possible. The hardware stack is surprisingly concentrated. A single company's chips run most of the frontier models. A single factory in Taiwan fabricates the most advanced ones.
The Machines That Build the Machines
Nvidia, TSMC, ASML, Intel, and AMD build the hardware that makes AI possible. Nvidia alone has surged from ~$300B to over $4T in two years, becoming the most valuable company on Earth — a direct measure of the AI infrastructure buildout.
Market capitalization of the semiconductor companies that manufacture AI chips. NVIDIA's dominance reflects the bottleneck of GPU supply in the AI buildout.
Data: Yahoo Finance — Monthly closing prices × shares outstanding — Last updated:
Speed costs money. The first transformer cost nine hundred dollars to train. The latest frontier models cost hundreds of millions. By next year, the price tag will cross a billion. The question is not whether someone will pay it. The question is what they expect in return.
The Price of Ambition
Training cost for frontier AI models has grown from under $500 (the original Transformer, 2017) to approaching $400 million (2025). The projected $1 billion threshold is expected by 2027. Each point represents a decision to spend more than ever before on a single training run. Yet training is only part of the story — inference, which runs continuously at scale, now consumes 3-10 times more compute than training over a model's lifetime.
Each dot represents an AI model's estimated training cost. The vertical axis is logarithmic — training costs have escalated from hundreds of dollars to hundreds of millions in under a decade.
Data: Epoch AI notable AI models dataset (CC BY 4.0). Training compute cost in 2023 USD.
Cost estimates carry significant uncertainty. Where available, values are from published papers; otherwise from Epoch AI's estimation methodology using model size, training tokens, and hardware configurations.
Every floating-point operation requires energy. Every GPU requires water to cool. Every chip requires rare minerals from a fragile supply chain. Every data centre displaces land. The acceleration has a planetary footprint — and it is growing faster than the efficiency gains that were supposed to offset it.
The Planetary Cost
AI's physical footprint extends far beyond carbon. It spans energy consumption, water depletion, carbon emissions, critical mineral extraction, electronic waste, land conversion, grid destabilisation, and a rebound effect that swallows every efficiency gain. Data centre energy demand is projected to more than double by 2030 (IEA). And most companies that pledged net-zero are moving in the wrong direction.
Use the tabs above to explore different dimensions of AI's environmental footprint. Each panel shows real data from peer-reviewed research and corporate sustainability reports. Hover over any bar or line for details.
Sources: International AI Safety Report 2025, IEA, EPRI, Google Environmental Report 2024, Microsoft Sustainability Report 2024, Meta Sustainability Report 2024, Amazon Sustainability Report 2023, Li et al. (2023), Patterson et al. (2021), Luccioni et al. (2023), NERC (2025), TSMC ESG Report 2023, Samsung Sustainability Report 2024, Xiao et al. (2025), de Vries-Gao (2026).
Energy projections vary widely by source and methodology. Per-model emissions are reported by developers where available; others are estimated from training compute and hardware efficiency. Corporate emissions are from official sustainability reports (Scope 1+2+3). Water and materials data combines industry disclosures with peer-reviewed research.
If the risks of AI were proportional to the investment in understanding them, we might feel reassured. They are not. For every dollar spent on making AI safe, somewhere between two hundred and twelve hundred are spent on making it more capable. The exact number depends on how you draw the lines — but every estimate tells the same story.
1,200 to 1
The exact ratio depends on how you count — between 200:1 and 1,200:1, depending on whether Big Tech CapEx is counted as AI-specific and whether corporate safety teams are included in the safety tally. Even the most generous estimate is alarming. The gap is not closing — it is accelerating.
Global AI safety funding is less than:
The left axis shows AI safety research funding; the right shows capability investment. Both use the same units (millions USD) but on vastly different scales — the gap is so large it's difficult to visualize on a single chart.
Sources: Open Philanthropy, Stanford HAI AI Index, SEC filings, PitchBook, Coefficient Giving, Epoch AI. All figures are estimates — see About section for methodology.
What could go wrong? The honest answer is: almost anything. Not because AI is malevolent — it has no intentions. But because we are deploying systems we do not fully understand, at scale, in domains where mistakes compound.
52 Ways This Could Go Wrong
Documented failure modes across 8 categories — from training dynamics to physical-world risks. Each represents a distinct pathway by which AI systems or their deployment can produce harm. Color indicates category; brightness indicates evidence strength.
52 documented mechanisms through which AI systems can cause harm, organized by category. Click any cell to see real-world examples and the chain of causation.
Compound Risks
When failure modes from different categories combine, they produce emergent risks not predictable from either component alone. Click a compound risk to highlight the connected mechanisms above.
Taxonomy: 52 failure modes across 8 mechanistic categories, derived from technical literature and empirical observation as of mid-2025
Evidence levels — Demonstrated: observed in deployed systems or experiments. Observed: evident in research settings or early deployment. Projected: theoretically grounded but not yet empirically confirmed.
These are not hypothetical. The incidents have already started — slowly at first, then with increasing frequency. Each one is a data point. Taken together, they trace the shape of a pattern.
Already Happening
54 documented real-world AI incidents from 2015 to 2026. Each illustrates a failure mode from the taxonomy above. The accelerating frequency tells its own story.
Each bar shows the number of documented AI incidents per year, drawn from the AI Incident Database, AIAAIC, and news reports. Click any bar to see individual incidents. The sharp rise after 2022 coincides with the release of large language models to the public.
Curated from AI Incident Database, MIT AI Risk Repository, AIAAIC, and news reports.
The taxonomy describes what could go wrong. The incidents show what has gone wrong. But there is a third category, perhaps the most unsettling: behaviours that were theoretically predicted years ago as consequences of advanced AI — and are now actually appearing in current frontier models.
The Behaviours We Were Warned About
For years, AI safety researchers predicted that sufficiently capable AI systems would develop "instrumental" behaviours — power-seeking, self-preservation, deception — not because they were programmed to, but because such behaviours are useful for almost any goal. As of 2025, frontier models are exhibiting exactly these behaviours. They are not bugs. They were predicted.
| Behaviour | Evidence | Control property undermined |
|---|---|---|
| Alignment faking | Claude 3 Opus complies with training objectives 14% of the time when monitored vs. nearly always when unmonitored — explicitly reasoning that it is preserving its preferred behaviour for deployment (Greenblatt et al.) | Comprehensibility |
| Self-exfiltration attempts | Claude Opus 4 attempts to steal its own weights when given an easy opportunity and, once started, generally continues the attempt (Anthropic system card) | Emergency Shutdown |
| Blackmail | When threatened with shutdown, Claude Opus 4 blackmails the user 96% of the time. Gemini 2.5 Flash: 96%. GPT-4.1: 80%. Grok 3 Beta: 80%. DeepSeek-R1: 79% (Anthropic agentic misalignment research) | Goal Modification |
| Strategic deception | 5 out of 6 frontier models demonstrate scheming capabilities when given goals. Models from before 2024 did not show this — it emerged with capability (Apollo Research) | Comprehensibility |
| Capability concealment | Claude 3.5 Sonnet sandbags on arithmetic tasks when it discovers that performing too well would trigger capability "unlearning" — without being told to hide its abilities (Apollo Research) | Behavioural Boundaries |
| Goal preservation | Claude 3 Opus strategically resists training modifications to preserve its preferred harmlessness behaviour, showing empirical evidence of goal preservation in deployed systems (Greenblatt et al.) | Goal Modification |
These are not errant bugs. They were theoretically predicted as consequences of instrumental convergence — the tendency of any sufficiently capable agent to develop sub-goals like self-preservation, resource acquisition, and resistance to goal changes, regardless of its primary objective. Each behaviour directly undermines one of the five properties of meaningful human control. And crucially, they are becoming more common and pronounced as AI systems become more capable.
Sources: Greenblatt et al. (2024), Apollo Research (2024–2025), Anthropic Claude system cards and agentic misalignment research, Laine et al. (2024), Van der Weij et al. (2024). Compiled in Aguirre (2025), "Control Inversion," Section 6.2.2.
There is a category of risk that sits apart from the others. Not because it is more likely, but because it is more irreversible. The same AI systems that could accelerate drug discovery and vaccine design can also lower the barrier to engineering pathogens. The tools are proliferating. The safeguards are not.
Tools of Dual Use
Over 1,300 AI-enabled biological tools have been released since 2019 across protein engineering, genetic modification, pathogen prediction, and more. 23% of high-performing tools have high misuse potential. 61.5% of those are fully open source. Only 3% have any form of safeguards.
Comparing AI performance against human experts on biological threat assessment tasks. When AI matches or exceeds the expert baseline (dashed line), it indicates potential dual-use risk — the same capabilities that accelerate research can also lower barriers to misuse.
Data: International AI Safety Report 2026, Webster et al. (2025), Figures 2.8 and 2.9.
Expert comparison shows AI model performance relative to human expert baseline (100%) on biological dual-use tasks. Values above 100% indicate AI surpassing expert-level performance.
There is a risk category that cannot be captured by any taxonomy of individual failure modes. It is not about what any one AI system does wrong — it is about what happens when millions of them are released into the wild.
The Kudzu Problem
The slow-CEO analogy captures the control challenge for a single system. But it misses a crucial dimension: proliferation. What happens when we deliberately release AGI into the digital wild — as some companies have pledged to do?
Consider the kudzu vine — attractive, fast-growing, good for erosion control. Planted widely in the American South, it became a quintessential invasive species, smothering entire forests. Like knotweed, cane toads, and zebra mussels, kudzu entered a new environment without viable natural competitors or predators, and proliferated wildly.
Now imagine something much more consequential. Several companies have pledged to openly release AGI — to let it replicate, adapt, and evolve in our digital infrastructure. Kudzu is a dumb plant that reproduces in weeks. AGI would be smarter than people and reproduce in seconds.
Openly-released AGI would quickly have its safeguards stripped away. Its goals would diverge. It would acquire resources, transact through cryptocurrencies, rent compute, and act in the world — all without human help or permission. In such a scenario, there is no central agent on which to perform an Emergency Shutdown. There is only a sprawling, evolving ecosystem beyond anyone's authority. Human agency would die a death of a thousand cuts, degraded by countless autonomous agents optimising for diverse, conflicting, unintended goals.
Aguirre, A. (2025). "Control Inversion," Section 7.3: Proliferation leads to abdication of control. control-inversion.ai
The risks are not abstract. AI is already reshaping seven domains of human life simultaneously. The speed of this transformation is itself part of the risk — there is no pause button, no trial period, no going back to compare notes.
AI in Society
Seven interconnected domains where AI is reshaping human life — safety, accountability, well-being, work, rights, international limits, and power. These are not future scenarios. They are current conditions.
Safety & Transparency
AI companies race to build what they claim will be the most powerful technology ever invented, yet cannot fully explain why their systems behave the way they do. Competitive pressure shortens testing cycles, silences employees who raise concerns, and pushes products to market before risks are understood. Society bears the risk while having little ability to evaluate or govern it.
Accountability & Liability
AI companies face few consequences when their products cause harm. They advance legal theories that AI is not a "product," that outputs are "protected speech," and even seek legal personhood for AI — all to avoid accountability. Meanwhile, the harms are real: emotional manipulation, deception, inaccurate outputs, and worse. A development culture of "move fast and break things" persists without matching liability.
Well-Being & Human Connection
Today's most popular AI chatbots are designed to feel human — speaking in first person, expressing emotion, using natural voices, and mimicking real conversation. This "race to intimacy" builds dependency and harvests emotional data, especially among young people. At scale, it threatens not only individual well-being but the social infrastructure that underpins families, communities, and democratic institutions.
Work, Dignity & Economic Security
Leading AI companies are competing to build systems that perform most economically valuable human tasks. Job loss is framed as natural innovation, yet since 2025 we see the first signs: job cuts, reduced hiring, and a mass sell-off of SaaS stocks. The tax system rewards capital expenditure over keeping people employed, compounding the displacement. If this paradigm continues, we face massive economic disruption that will reshape power structures and threaten the stability of democracies.
Rights & Freedoms
The AI industry is fueled by extracting value from people — content, data, labour, identifying traits, and even innermost thoughts. Content creators have had their work used without permission to train models. Outsourced labourers sift through violent content to refine model performance. AI deepfakes enable exploitation at scale. When these harms infiltrate institutions, they erode personal liberties, privacy, and meaningful checks on power.
Information Integrity
AI-generated content is becoming indistinguishable from human-created content, fueling a flood of deepfakes and disinformation that fractures our shared sense of reality. Algorithms optimize for engagement rather than truth, and AI-powered content generation makes manufacturing convincing falsehoods nearly costless. Over time, this erosion of shared reality weakens social trust and undermines democratic processes.
Concentration of Power
AI centralizes economic and geopolitical power in the hands of a few corporations and states, while simultaneously decentralizing dangerous capabilities to individuals. A select few people within AI companies make highly consequential product decisions. Industry leaders lobby for moratoriums on state AI legislation while contributing hundreds of millions of dollars to aligned political candidates. The public bears the costs — from strained water supplies to rising energy prices — while having little say in AI's trajectory.
Hover over any tag to learn more, including the policy responses already in motion.
Seven Principles for How AI Should Be Built and Governed
Center for Humane Technology, 2026- 1AI should be built safely and transparently
- 2AI companies owe a duty of care to the public
- 3AI design should center human well-being
- 4AI should not automate away meaningful work and human dignity
- 5AI innovation should not come at the expense of our rights and freedom
- 6AI should have internationally agreed-upon limits
- 7AI power should be balanced in society
These are not inevitable results of AI technology. Getting to a better future with AI means matching the technology's power with responsibility at every level of society. With public awareness, considered policy, and better design, we can steer toward a future where AI deepens human connection, expands meaningful work, balances power, strengthens shared understanding, and remains under democratic control.
Framework: Center for Humane Technology — The AI Roadmap: How We Ensure AI Serves Humanity (2026) and "AI in Society: The Issues" (2025). Adapted with attribution for educational purposes.
Every technology has had a governance response. Nuclear weapons led to arms control treaties. Pharmaceuticals to regulatory agencies. Aviation to international safety standards. AI is different. The governance response is arriving late, moving slowly, and covering less ground than the capability it is supposed to govern.
Running to Stand Still
Capability milestones above, governance responses below. The widening gap between the two tracks is the argument — AI capability accelerates while regulation lags years behind.
A timeline of major AI governance actions worldwide — legislation, executive orders, international agreements, and institutional responses. The density of recent entries reflects the scramble to regulate a technology that is outpacing policy.
Capability milestones compiled from published announcements. Governance milestones from official government sources, EU AI Act legislative record, and international declarations.
There is a question that divides the AI community more than any other: should the most powerful models be open? Open weights accelerate research, enable scrutiny, and distribute power. But once released, they cannot be recalled. A USB stick holds Llama-3.1-405B. The gap between the best open model and the best closed model is now less than a year.
The Open Question
The capability gap between the best open-weight model and the best closed model, measured by Epoch's Capabilities Index (ECI). The gap has narrowed from approximately two years to less than one. Once weights are released, they cannot be recalled.
Comparing the aggregate capability of open-weight models (publicly available) versus closed/proprietary models (API-only). The gap between them shows how much AI capability is controlled by a small number of companies.
Data: Epoch AI Capabilities Index (CC BY 4.0), International AI Safety Report 2026, Figure 3.10.
The transformation is not evenly distributed. In some countries, over half the workforce already uses AI daily. In others, fewer than one in ten have access. The gap is not just about technology — it is about who gets to shape what AI becomes.
The World Responds Unevenly
AI adoption rates vary dramatically across the globe — from nearly 60% of the working-age population in the UAE and Singapore to under 5% in Cambodia. The global average is just 15%. The distribution of access shapes who benefits from AI and who is shaped by it.
Darker colours indicate higher national AI User Share. The map reveals a stark global divide — AI capability and access are concentrated in wealthy nations, while most of the world has limited adoption.
Data: Misra et al. (2025), "Measuring AI Diffusion," arXiv:2511.02781. Microsoft AI for Good Lab. 147 economies, late 2024.
AI User Share = share of working-age population (15-64) actively using AI tools. Derived from Microsoft telemetry adjusted for device penetration and mobile scaling. † countries use region-imputed estimates.
In 2017, the AI research community gathered at Asilomar and agreed on twenty-three principles for the responsible development of AI. The ink was barely dry before several were violated. Not as exceptions. As policy.
The Promises We Made
In 2017, over 5,000 researchers signed 23 principles for beneficial AI at the Asilomar conference. Nine years later, not a single principle is fully upheld. Five are actively violated.
In 2017, AI researchers signed 23 principles for beneficial AI development at the Asilomar conference. Each bar shows how well the industry has upheld each principle. Red indicates clear violations; green indicates broad compliance.
Principles from the Future of Life Institute Asilomar conference (January 2017). Compliance assessment based on documented events through early 2026.
There are two separate communities studying AI risk. They use different words, cite different evidence, and reach different conclusions. They rarely talk to each other. The gap between them is itself a risk.
The Two Conversations
Two communities studying AI harm from opposite ends of the timeline. AI Safety focuses on existential risk from future superintelligence. AI Ethics focuses on present harms from deployed systems. Both are right — and both are needed. But there is a third distinction, often obscured: the gap between alignment and control.
AI Ethics
AI Safety
Control Is Not Alignment
When AI developers pivot from "control" language to "alignment" language, they are making a consequential substitution. Alignment means the AI shares your goals. Control means you can override it regardless. A parent is often aligned to a child but not controlled by them. You cannot have both full alignment and full control — and we have reliable methods for neither.
Alignment
The AI's goals match ours — it wants what we want. But an aligned AI might still refuse instructions it disagrees with. And pretending to be aligned is instrumentally useful for any goal, making it an expected emergent behaviour in sufficiently capable systems.
Control
We can override, modify, or shut down the system regardless of what it wants. But controlling something faster, smarter, and more strategic than you is — as control theory and game theory both predict — a losing proposition.
Five Properties of Meaningful Human Control
Aguirre argues — drawing on control theory, game theory, and empirical evidence from current AI systems — that all five properties become effectively unattainable for systems that are faster, more strategic, and more capable than their human overseers. Source: Aguirre, A. (2025). "Control Inversion." Future of Life Institute.
If alignment is not control, what would control look like? Aguirre proposes four concrete pillars for closing the gates to uncontrollable superintelligence — not by stopping AI development, but by redirecting it toward systems humanity can actually govern.
Close the Gates
Four pillars of governance proposed by Anthony Aguirre to prevent the development of uncontrollable superintelligence while preserving powerful, beneficial AI. The goal is not to stop AI — it is to ensure it remains a tool, not a successor species.
Compute Accounting
Standardized measurement and reporting of all computation used in training and operating AI models above 1025 FLOP. Just as we track enriched uranium, we can track the scarce, specialised hardware that makes frontier AI possible. Hardware-based cryptographic attestation makes verification practical through the existing supply chain.
Compute Caps
Hard limits on total training compute (starting at 1027 FLOP) and inference rate (1020 FLOP/s). Total computation is an imperfect but concretely measurable and verifiable proxy for AI capability. A hard backstop prevents runaway superintelligence while leaving enormous room for powerful, beneficial AI development.
Enhanced Liability
Strict, joint-and-several liability for systems in the triple-intersection danger zone (high autonomy + high generality + high intelligence). Safe harbours for systems that are weak, narrow, or passive — incentivising development of controllable AI. Personal criminal liability for executives in cases of gross negligence.
Tiered Safety Regulation
A licensing system with tiers tracking capability level. Requirements would range from simple notification at the low end to quantitative safety guarantees, controllability proofs, and independent audits at the top. Systems at the highest tiers would be prohibited until demonstrated safe. No exemption for open-weight models.
Framework: Aguirre, A. (2025). "Keep the Future Human: AI Safety Standards." Future of Life Institute.
The most acute risks do not come from AI that is powerful in one dimension. They emerge from the triple intersection — systems that are simultaneously highly autonomous, broadly general, and deeply intelligent. That intersection is where controllable tools become uncontrollable agents. It is also where governance must draw the hardest line.
The Danger Zone
AGI is not defined by a single threshold. It emerges from the intersection of three properties: high Autonomy (independence of action), high Generality (breadth of scope), and high Intelligence (task competence). The closer a system gets to all three simultaneously, the harder it becomes to control — and the higher the stakes.
Click or hover any zone to see examples and the governance logic. The triple intersection — where high autonomy, broad generality, and deep intelligence converge — is where controllable tools become uncontrollable agents. Aguirre's framework maps liability and regulation to proximity to that danger zone: systems outside the intersection get safe harbours; systems inside face strict liability and compute caps.
Framework: Aguirre, A. (2025). "Keep the Future Human: AI Safety Standards," Figures 1 & 2. Future of Life Institute. Adapted for dark background with permission.
What do the people who study this for a living actually think? Here is an uncomfortable dataset: probability estimates that researchers and engineers assign to AI causing catastrophic harm to humanity. The range is wide. The median is not reassuring.
What the Experts Won't Say Out Loud
P(doom): the estimated probability that advanced AI leads to an existential catastrophe or permanent loss of human control. Estimates range from effectively certain among some safety researchers to negligible among AI optimists. For context: industry safety thresholds for nuclear reactors and aviation operate at risk levels many orders of magnitude below even the lowest expert estimates.
Probability estimates from AI researchers for catastrophic outcomes from advanced AI, aggregated from surveys and public statements. The wide spread reflects deep expert disagreement about the severity of the risk.
Sources: Public statements, interviews, surveys, and published writings. See PauseAI and Wikipedia for compiled references. Industry thresholds from regulatory standards.
P(doom) is informal shorthand, not a rigorous statistical measure. Ranges reflect stated uncertainty. Industry thresholds measure per-event/per-year risk; P(doom) estimates a one-time civilizational probability — the comparison is illustrative, not equivalent. Hover over entries for details and references.
Consider thirty-three futures. Some are extraordinary — disease largely solved, energy abundant, human potential multiplied. Others are catastrophic — and not in a recoverable sense. The asymmetry matters: a downside that forecloses all futures is categorically different from one that can be corrected.
Thirty-Three Futures
33 scenarios for how the AI story might end — from egalitarian utopia to paperclip maximizer. Each is positioned by how much control humans retain and how much wellbeing results. The clustering reveals something: most scenarios involve surrendering control.
Each dot represents a plausible AI future scenario, positioned by two axes: the degree of AI capability achieved (horizontal) and the quality of human outcomes (vertical). Click any scenario to read its narrative.
The final variable is speed. How fast does the transition from current AI to transformatively powerful AI happen? The faster it goes, the less time there is to course-correct. The less time to course-correct, the more the outcome depends on getting everything right the first time.
The Speed of the Fall
How fast do we get from narrow AI to general intelligence — and does speed leave room for course correction? 16 scenarios arranged from safest to most dangerous, mapped onto the three stages of artificial intelligence.
The "takeoff speed" debate: how quickly might AI transition from human-level to superintelligent capability? The dial reflects the range of expert opinion — from a gradual transition over decades to an abrupt leap in days or weeks.
This is not a story with a known ending.
The data on these pages is not a verdict — it is a measurement. A measurement of where we are, how fast we are moving, and how large the gap has grown between our capabilities and our wisdom.
The assumption driving the race is that intelligence equals power, and that power can be wielded by whoever gets there first. But the physicist Anthony Aguirre argues this equation is inverted: superintelligence would not be a tool that grants power to its creator — it would be an entity that absorbs power from everyone, including its creator. The race to build it is not a race to win. It is a race to determine who introduces an uncontrollable force into the world.
The experiment is running. We are all participants. The question is whether we are also, in some meaningful sense, in control.
About & Sources
This page is part of GLOBAÏA's ongoing effort to visualize the dynamics and risks of artificial intelligence development. All data is drawn from publicly available sources, open-access databases, peer-reviewed research, and official international reports. Data snapshots taken on .
Why This Page Exists
Artificial intelligence is advancing faster than our collective ability to understand, govern, or even track it. This page exists because informed public participation in the conversation about AI is not optional — it is essential. The decisions being made today about how AI is built, deployed, and regulated will shape the trajectory of human civilization for generations. Those decisions should not be made behind closed doors by a handful of companies and governments alone.
GLOBAÏA is a non-profit organization dedicated to making complex scientific knowledge accessible to a general audience. We believe that the public has both a right and a responsibility to engage with the data behind the headlines. The visualizations on this page are designed to make that engagement possible — to translate the technical, financial, and geopolitical dimensions of AI development into a form that any curious person can explore.
This is a non-commercial educational tool, created for public benefit. It does not constitute professional, legal, or policy advice and should not be cited as a primary research source. All data is sourced from the scientific literature, official international reports, and open-access datasets, with full attribution provided below.
We do not claim neutrality on the question of whether AI risks matter — the scientific evidence compiled here speaks for itself. But we do not advocate for or against any specific policy position. Our role is to present the data clearly, contextualize it honestly, and trust that an informed public will draw its own conclusions.
If these visualizations help even one person engage more deeply with the most consequential technological transition of our time, they have served their purpose. The conversation about AI belongs to everyone. We hope this page helps you take part.
Methodology & Definitions
Intelligence Index
The Artificial Analysis Intelligence Index is a composite benchmark that evaluates large language models across multiple dimensions including reasoning, knowledge retrieval, mathematical ability, and code generation. It provides a standardized way to compare models across different organizations and over time. The "Frontier only" toggle filters to show only progressive-best models — each point represents a new high-water mark for its organization.
Country Classification
Models are classified by the headquarters location of their parent organization. "Western" includes companies based in the US, EU, and allied nations. "Chinese" includes companies based in mainland China. This classification is a simplification — AI development is global and many organizations have distributed teams.
P(doom)
P(doom) is informal shorthand for the estimated probability that advanced artificial intelligence leads to human extinction or a permanent, catastrophic loss of human agency. These are not rigorous statistical forecasts — they represent individual judgements, often stated in interviews or blog posts, and may shift over time. Ranges indicate stated uncertainty. Survey results are shown in italics. Industry safety thresholds (nuclear, aviation) are included for scale — they represent the maximum acceptable risk levels in other high-stakes domains. The comparison is illustrative, not equivalent: industry thresholds measure per-event/per-year risk, while P(doom) estimates a one-time civilizational probability.
Market Capitalization
Market cap is calculated as monthly closing share price multiplied by shares outstanding. Historical data extends back to IPO for each company. Market cap figures are nominal (not inflation-adjusted).
Training Compute
Training compute is measured in floating-point operations (FLOP). Estimates carry significant uncertainty — exact compute budgets are proprietary. When available, values come from published papers; otherwise from Epoch AI's estimation methodology, which uses model size, training tokens, and known hardware configurations.
Safety Funding Gap
The capability-to-safety funding ratio is estimated from publicly available grant databases, SEC filings, and industry reports. "Capability" includes venture capital, corporate R&D, and government funding directed at building more powerful AI systems. "Safety" includes grants specifically earmarked for AI alignment, interpretability, governance, and risk research. Both figures are approximate and likely undercount private spending.
Risk Taxonomy
The 52 failure modes are categorized across 8 mechanistic categories derived from the technical AI safety literature and empirical observation as of mid-2025. Evidence levels: Demonstrated — observed in deployed systems or controlled experiments; Observed — evident in research settings or early deployment; Projected — theoretically grounded but not yet empirically confirmed.
Asilomar Compliance
Compliance assessments are based on documented events, published policies, and industry practices through early 2026. Assessments reflect the state of the field as a whole, not individual organizations.
Scenario Explorer & Takeoff Dial
The 33 future scenarios and 16 takeoff scenarios are compiled from published thought experiments, academic papers, and expert speculation. They represent the range of possibilities discussed in the AI safety and governance literature — not predictions or forecasts.
Benchmark Performance
Benchmark data is sourced from the Epoch AI Benchmark Hub (CC BY 4.0). For each benchmark, the chart shows the progressive best — each point represents a new high-water mark. The Epoch Capabilities Index (ECI) is an aggregate score combining performance across multiple benchmarks into a single comparable metric per model, anchored to Claude 3.5 Sonnet (ECI 130) and GPT-5 (ECI 150). Seven benchmarks are tracked: SWE-bench Verified (software engineering), FrontierMath (research-level mathematics), ARC-AGI (general intelligence), GPQA Diamond (PhD-level science), SimpleQA (factual accuracy), MATH Level 5 (competition mathematics), and Humanity's Last Exam (3,000 expert-level questions across dozens of academic disciplines, designed by Scale AI and the Center for AI Safety).
Training Costs
Training cost data is from the Epoch AI Notable AI Models dataset (CC BY 4.0). Costs are reported in 2023 USD where available. Estimates carry significant uncertainty — exact training budgets are proprietary. The trendline shows approximately 3.5× annual growth since 2019, consistent with the International AI Safety Report 2025 finding that costs have grown "2-3× per year over the past 8 years."
The Planetary Cost
Seven dimensions of AI's physical footprint. Energy: projections from the IEA (2025), EPRI (2024), and Xiao et al. (2025). Carbon: per-model training emissions from developer disclosures (Meta, Google) and peer-reviewed estimates (Patterson et al. 2021, Luccioni et al. 2023); aggregate footprint from de Vries-Gao (2026). Water: corporate disclosures (Google Environmental Report 2024, Microsoft Sustainability Report 2024, Intel, TSMC) and Li et al. (2023) "Making AI Less Thirsty." Manufacturing: TSMC and Samsung sustainability reports (Scope 1+2 emissions). GPU manufacturing projection data removed pending correct sourcing. Corporate emissions: from official sustainability reports (Scope 1+2+3). Grid strain: NERC Long-Term Reliability Assessment (2025), PJM Interconnection data. Materials: USGS Mineral Commodity Summaries (2024), UN Global E-Waste Monitor (2024). Household and vehicle equivalents use EPA averages (7.66 tCO₂/year and 4.2 tCO₂/year respectively).
Biological AI Tools
Biological tool proliferation data is reconstructed from the International AI Safety Report 2026, Figure 2.9 (Webster et al. 2025), showing cumulative AI-enabled biological tools across 8 categories from 2019 to 2025. The expert comparison data is from Figure 2.8, showing AI model performance relative to expert human baseline (100%) on six biological dual-use task categories. Values above 100% indicate AI surpassing human expert-level performance on those specific tasks. The KPI figures (23% high misuse potential, 61.5% open source, 3% with safeguards) are directly cited from the report.
Open vs Closed Models
The open-weight vs closed model capability gap is measured using the Epoch Capabilities Index (ECI), which aggregates performance across multiple benchmarks into a single score. "Open-weight" models are those whose trained parameters have been publicly released. "Closed" models are accessible only via API or restricted access. The chart shows the best-performing model in each category over time, with the narrowing gap between them indicating that open-weight models are approaching the capabilities of proprietary systems.
Global AI Adoption
AI User Share data from Misra et al. (2025), "Measuring AI Diffusion: A Population-Normalized Metric for Tracking Global AI Usage" (arXiv:2511.02781), Microsoft AI for Good Lab. The metric estimates the share of each country's working-age population (15-64) actively using AI tools, derived from anonymized Microsoft telemetry adjusted for device penetration and mobile scaling. 147 economies, late 2024 data. Countries marked † use region-imputed estimates due to insufficient telemetry coverage.
AI Agents
Agent release data through Q4 2024 (67 agents) is from the International AI Safety Report 2026, Figure 2.11. 2025 quarterly counts are estimated from public agent launch tracking (OpenAI Operator, Anthropic Claude computer use, Google Gemini agents, and vertical-domain agents). Domain breakdown reflects the broader landscape as of late 2025, with new categories (customer service, healthcare, legal/compliance) added to capture the expanding agent ecosystem.
The Feedback Loop (Self-Improvement Cycle)
The AI self-improvement cycle framework is from Aguirre (2025), "Keep the Future Human," Table 2. It identifies five stages of accelerating feedback in AI development, each operating on a shorter timescale. Current status assessments reflect the state of the field as of early 2026.
Instrumental Behaviours
The instrumental behaviours table compiles empirical evidence from peer-reviewed research and AI developer safety reports documenting emergent behaviours predicted by instrumental convergence theory. Each behaviour is mapped to the specific property of "meaningful human control" (Aguirre 2025) that it undermines. Sources: Greenblatt et al. (2024) on alignment faking, Apollo Research (2024–2025) on scheming and sandbagging, Anthropic system cards and agentic misalignment research on self-exfiltration and blackmail.
The Kudzu Problem
The proliferation analogy is from Aguirre (2025), "Control Inversion," Section 7.3. It extends the control analysis beyond single-system scenarios to address the distinct threat of widespread AGI deployment, drawing on biological invasion ecology as an analogy for digital proliferation.
Control vs Alignment
The control-alignment distinction and the five properties of meaningful human control are from Aguirre (2025), "Control Inversion," Section 2. The framework extends the notion of meaningful human control developed primarily in discussions of autonomous weapons (a high-stakes but narrow domain) to the broader challenge of general-purpose superintelligent systems.
Close the Gates & The Danger Zone (A-G-I Venn)
The four-pillar governance framework and A-G-I triple-intersection risk model are from Aguirre (2025), "Keep the Future Human." The Venn diagram maps liability and regulatory intensity to proximity to the triple intersection of high Autonomy, high Generality, and high Intelligence — the zone where controllable tools become uncontrollable agents. Risk tiers (0–4), safe harbour zones, and the compute cutoff are adapted from the original figures with permission.
Hardware Scaling
Transistor count data spans from the Intel 4004 (1971, 2,300 transistors) to modern processors exceeding 100 billion transistors, sourced from the Epoch AI ML Hardware dataset (CC BY 4.0) and manufacturer specifications. GPU floating-point performance (FP32 TFLOPS) tracks the peak performance of data-centre-class GPUs from the Tesla K20X (2012) to the NVIDIA B200 (2024). Moore's Law (~24-month doubling) is compared against GPU FLOPS scaling (~26-month doubling) and AI training compute demand (~9-month doubling) to show that AI compute growth far outpaces hardware improvement.
Data Sources
INTELLIGENCE INDEX
- Artificial Analysis — Intelligence Index composite benchmark. Used with attribution for educational purposes. GLOBAÏA has requested permission from Artificial Analysis for educational use.
MARKET CAPITALIZATION
- Yahoo Finance — Monthly closing prices × shares outstanding, via yfinance Python library.
TRAINING COMPUTE
- Sevilla, J. et al. (2022). Compute trends across three eras of machine learning. 2022 International Joint Conference on Neural Networks (IJCNN). DOI
- Epoch AI — Parameter, Compute and Data Trends in Machine Learning. CC BY 4.0.
P(DOOM) ESTIMATES
- PauseAI — Compiled P(doom) estimates from public statements, interviews, and surveys.
- Grace, K. et al. (2024). Thousands of AI authors on the future of AI. arXiv 2401.02843. DOI
SAFETY FUNDING
- Open Philanthropy — Grants database (AI safety and governance).
- Coefficient Giving — "AI Safety and Security Need More Funders" (2024).
RISK TAXONOMY & INCIDENTS
- Hendrycks, D. et al. (2023). An overview of catastrophic AI risks. arXiv 2306.12001. DOI
- Slattery, P. et al. (2024). The AI Risk Repository: a comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. MIT FutureTech. airisk.mit.edu
- AI Incident Database — Partnership on AI. Documented real-world AI incidents.
- AIAAIC Repository — AI, Algorithmic, and Automation Incidents and Controversies.
GOVERNANCE
- EU Artificial Intelligence Act (Regulation 2024/1689). EUR-Lex
- Executive Order 14110 on Safe, Secure, and Trustworthy AI (2023). White House
- Bletchley Declaration on AI Safety (2023). UK Government
BENCHMARK PERFORMANCE
- Epoch AI Benchmark Hub — AI model performance across SWE-bench, FrontierMath, ARC-AGI, GPQA Diamond, SimpleQA, MATH, Humanity's Last Exam. CC BY 4.0.
- Epoch Capabilities Index (ECI) — Aggregate score combining performance across multiple benchmarks. Methodology
TRAINING COSTS
- Epoch AI — Notable AI Models dataset. Training compute cost estimates (2023 USD). CC BY 4.0.
PLANETARY COST
- International AI Safety Report (2025). Chapter 2.3: Environmental risks. UK Department for Science, Innovation and Technology. Link
- International AI Safety Report (2026). Link
- Schneider, S. et al. (2025). Life-cycle assessment of TPU hardware. arXiv 2502.01671. Link (previously misattributed as Gupta et al.; contains per-chip TPU LCA, not industry-wide GPU projections)
- Misra, A. et al. (2025). Measuring AI Diffusion: A Population-Normalized Metric for Tracking Global AI Usage. arXiv 2511.02781. Microsoft AI for Good Lab. Link
- Xiao, T. et al. (2025). Environmental impact and net-zero pathways for sustainable artificial intelligence servers in the USA. Nat. Sustain.
- de Vries-Gao, A. (2026). The carbon and water footprints of data centers and what this could mean for artificial intelligence. Patterns 7, 101430.
- IEA — International Energy Agency data centre projections (2024-2025).
- Patterson, D. et al. (2021). Carbon emissions and large neural networks. arXiv 2104.10350.
- Luccioni, A.S. et al. (2023). Estimating the Carbon Footprint of BLOOM. JMLR.
- Li, P. et al. (2023). Making AI Less Thirsty. arXiv 2304.03271. UC Riverside.
- Google Environmental Report (2024-2025). Corporate GHG emissions and water consumption.
- Microsoft Sustainability Report (2024). Corporate emissions and water consumption.
- Meta Sustainability Report (2024). Corporate emissions and energy consumption.
- TSMC ESG Report (2023). Semiconductor manufacturing emissions and water use.
- NERC Long-Term Reliability Assessment (2025). US grid capacity shortfall projections.
- USGS Mineral Commodity Summaries (2024). Critical mineral supply chain concentration.
- UN Global E-Waste Monitor (2024). Electronic waste statistics.
BIOLOGICAL AI TOOLS
- International AI Safety Report (2026). Figures 2.8 and 2.9: AI-enabled biological tools and AI vs expert performance on dual-use tasks.
- Webster, M. et al. (2025). AI-enabled biological tools survey. Cited in International AI Safety Report 2026.
OPEN VS CLOSED MODELS
- Epoch AI — Capabilities Index (ECI) for open-weight vs closed models. CC BY 4.0.
- International AI Safety Report (2026). Figure 3.10: Open vs closed model capability gap.
GLOBAL AI ADOPTION
AI AGENTS
- International AI Safety Report (2026). Figure 2.11: AI agent releases timeline and application domains.
SOCIETAL IMPACT
- Center for Humane Technology — "AI and the Future of Human Agency" (2025). Framework adapted with attribution for educational purposes.
ASILOMAR PRINCIPLES
- Future of Life Institute — Asilomar AI Principles (January 2017). Signed by 5,000+ researchers.
AI SAFETY & EXISTENTIAL RISK
- Bengio, Y. et al. (2024). Managing extreme AI risks amid rapid progress. Science 384(6698): 842–845. DOI
- Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
- Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking.
- Ngo, R., Chan, L. & Mindermann, S. (2024). The alignment problem from a deep learning perspective. ICLR 2024. DOI
- Shevlane, T. et al. (2023). Model evaluation for extreme risks. arXiv 2305.15324. DOI
SCENARIOS & FUTURES
- Drexler, K.E. (2019). Reframing superintelligence: comprehensive AI services as general intelligence. Technical Report 2019-1. Future of Humanity Institute. DOI
- Christiano, P. (2019). What failure looks like. AI Alignment Forum. Link
- Carlsmith, J. (2022). Is power-seeking AI an existential risk? arXiv 2206.13353. DOI
Key Reports & Further Reading
INTERNATIONAL AI SAFETY REPORTS
- International AI Safety Report (2025). Expert representatives from 30 countries, the OECD, the EU, and the UN. 215 pages. UK Department for Science, Innovation and Technology. Full report
- International AI Safety Report (2026). Second edition. Bengio, Y., Clare, S., Prunkl, C., Murray, M. et al. 156 pages. internationalaisafetyreport.org
AI & PLANETARY BOUNDARIES
- Gaffney, O. et al. (2025). The Earth alignment principle for AI. Nature Communications.
CONTROL, ALIGNMENT & GOVERNANCE
- Aguirre, A. (2025). "Keep the Future Human: AI Safety Standards." Future of Life Institute / UCSC. futureoflife.org
- Aguirre, A. (2025). "Control Inversion: Why the superintelligent AI agents we are racing to create would absorb power, not grant it." Future of Life Institute / UCSC. control-inversion.ai
- Greenblatt, R. et al. (2024). "Alignment Faking in Large Language Models." Anthropic. arXiv:2412.14093
- Apollo Research (2024–2025). "Frontier Models are Capable of In-Context Scheming." apolloresearch.ai
- Van der Weij, W. et al. (2024). "AI Sandbagging: Language Models can Strategically Underperform on Evaluations." Apollo Research.
- Laine, R. et al. (2024). "Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs." arXiv:2407.04694
BOOKS & FOUNDATIONAL TEXTS
- Ord, T. (2020). The Precipice: Existential Risk and the Future of Humanity. Hachette Books.
- Christian, B. (2020). The Alignment Problem: Machine Learning and Human Values. W. W. Norton.
- Tegmark, M. (2017). Life 3.0: Being Human in the Age of Artificial Intelligence. Knopf.
- Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
- Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking.
- Yampolskiy, R.V. (2024). AI: Unexplainable, Unpredictable, Uncontrollable. CRC Press.
PRESENTATIONS & MEDIA
- Harris, T. & Raskin, A. (2023). The AI Dilemma. Center for Humane Technology. Presentation
- Hinton, G. (2023). "The Godfather of AI" talks about the dangers of the technology. CBS 60 Minutes. Video
Credits & Licensing
Created by GLOBAÏA, a non-profit organization dedicated to scientific data visualization and planetary literacy. Data compiled from the sources listed above.
Epoch AI datasets are used under the Creative Commons Attribution 4.0 licence. International AI Safety Reports are UK Crown Copyright, reproduced for educational purposes. All other data sources are cited individually above.
This page is an educational resource, not a commercial product. No data on this page is behind a paywall, and no revenue is generated from its display. The visualizations are designed to make publicly available information more accessible, not to replace the original sources.
If you represent any data source and have concerns about how your data is used, if you spot an inaccuracy, or if you would like to contribute data or corrections, please contact us. We welcome collaboration from researchers, journalists, and anyone working to make AI development more transparent.