A field guide to the ideologies behind the AGI race
Silicon Valley is racing to build
a machine smarter than any human.
Ever wondered why?
Not how. Why. What do the people spending billions to build artificial general intelligence actually believe? In 2024, computer scientist Timnit Gebru and philosopher Émile P. Torres traced the answer to seven interlocking ideologies — and gave the bundle a name.
First, a strange fact
Nobody agrees on what they are building
Artificial general intelligence — AGI — is the stated goal of some of the best-funded organizations on Earth. It is also, remarkably, a system with no agreed definition. Ask its builders:
“Highly autonomous systems that outperform humans at most economically valuable work.”
“A software program that can solve a variety of complex problems in a variety of different domains, and that controls itself autonomously, with its own thoughts, worries, feelings, strengths, weaknesses and predispositions.”
“A computer system that matches or exceeds the real time cognitive (not physical) abilities of a smart, well-educated human.”
“A universal algorithm for learning and acting in any environment.”
Four builders, four different machines. Gebru & Torres argue this is not a detail: a system that is supposed to do any task under any circumstance cannot be tested for safety the way engineers test everything else — there is no defined job to test it against.
So why is building it treated as the obvious next step for humanity? To answer that, you have to meet seven ideologies. They emerged in roughly this order — and, as you'll see, they share founders, funders and a family tree.
A note before we start: not everyone working on AI — or even on AGI — subscribes to what follows. Many researchers are unaware of these ideologies entirely. The paper's claim, and this page's, is about the worldviews driving the race: the people who set its goals and write its cheques.
1957 — the word · late 1980s — the movement
Transhumanism
Humanity should upgrade itself
The belief that humanity can — and should — use technology to transcend itself: re-engineering our bodies and minds to become smarter, healthier and longer-lived, until we become something 'posthuman', as different from today's humans as we are from our distant ancestors.
- Radical 'human enhancement' — genetic engineering, brain implants, life extension — is both feasible and desirable.
- A 'posthuman' future awaits: indefinitely long 'healthspans', augmented minds, expanded senses.
- Humanity should take its own evolution into its hands.
“The human species can, if it wishes, transcend itself — not just sporadically … but in its entirety, as humanity.”
1988 — Extropy Institute
Extropianism
More intelligence, more order, forever
The first organized transhumanist movement — a libertarian variant built around 'extropy', the imagined opposite of entropy: intelligence, order and vitality expanding without limit, powered by technology and free markets.
- Boundless expansion: no natural limit to growth, intelligence or lifespan.
- Dynamic optimism: pessimism is a self-fulfilling prophecy; progress is a duty.
- Spontaneous order: free individuals and free markets, not governments, should steer the future.
“Extropy: the extent of a system's intelligence, information, order, vitality, and capacity for improvement.”
1991 — the word · 2005 — the bestseller
Singularitarianism
The intelligence explosion is coming
The belief that technological progress is accelerating toward a rupture in human history — the 'Singularity' — when self-improving artificial intelligence surpasses us, humans merge with machines, and change becomes too fast to comprehend.
- An 'intelligence explosion': AI improving itself in a runaway loop until it becomes 'superintelligent'.
- Humans and machines will merge; our descendants will 'wake up' the universe.
- The Singularity is near — Ray Kurzweil predicts 2045; Eliezer Yudkowsky once predicted 2025.
“We will transcend our frail bodies with all their limitations. Illness, as we know it, will be eradicated.”
2010 — A Cosmist Manifesto
Cosmism
Remake the universe itself
The most expansive ideology in the bundle: our posthuman descendants — sentient artificial intelligences, uploaded minds — should spread beyond Earth, build synthetic realities, and develop 'future magic' like spacetime engineering to transform the universe itself.
- Humans will merge with technology, opening 'a new phase of the evolution of our species'.
- Sentient AI and mind uploading will offer 'an indefinite lifespan to those who choose to leave biology behind'.
- We will 'spread to the stars and roam the universe', engineering reality as we go.
“We will develop spacetime engineering and scientific 'future magic' much beyond our current understanding and imagination.”
2009 — LessWrong
Rationalism
Think better, and AI is what matters
A community that formed around the blog LessWrong, dedicated to 'improving human reasoning and decision-making' — training minds with probability theory and bias-spotting. Many members concluded that the way to do the most good with a well-trained mind is to work on artificial intelligence.
- Human reasoning is riddled with biases and can be systematically trained.
- AI was identified early as 'a very big deal for the future of humanity'.
- Getting 'powerful AI outcomes to be good' is among the most important problems in the world.
“The LessWrong team … are predominantly motivated by trying to cause powerful AI outcomes to be good.”
2011 — Centre for Effective Altruism
Effective Altruism
Do the most good — by the numbers
A movement built on a compelling premise: use evidence and explicit calculation to do the most good possible with finite resources. It began with fighting global poverty. Then the calculations changed — if most people who will ever exist live in the far future, the math seems to point there instead.
- Good can be quantified, compared and maximized like any other quantity.
- 'Earning to give': it can be more ethical to get rich and donate than to help directly.
- Its career-advice arm now lists 'AI safety technical research' and 'AI governance' as the top causes.
“A candidate with a normal IQ of 100 would be subtracted PELTIV points, because points could only be earned above an IQ of 120.”
late 2010s — Oxford
Longtermism
The far future outweighs the present
An ethic that combines the rest of the bundle into a moral system: because the future could contain astronomical numbers of people — including trillions of digital people in computer simulations — positively shaping the far future matters more than nearly anything happening today.
- Nick Bostrom estimates 10⁵⁸ future digital people in the accessible universe — so even tiny changes in their odds outweigh everything else.
- 'Existential risk' — anything that would destroy this 'vast and glorious' potential — is the supreme concern.
- Becoming posthuman, colonizing space and maximizing 'value' are central to fulfilling our 'long-term potential'.
“We may simply ignore … the effects contained in the first 100 (or even 1,000) years.”
Where the threads come from
One root, seven branches
Seven ideologies, one family tree. Gebru & Torres trace all of them back to a single project: eugenics — the nineteenth-century ambition to engineer a “better” human stock. Follow the timeline; watch each thread split off.
- 1869 Eugenics
Francis Galton publishes Hereditary Genius
Galton argues that 'a man's natural abilities are derived by inheritance' — and that a 'highly-gifted race of men' could be produced 'by judicious marriages during several consecutive generations'. The 'scientific' groundwork for eugenics.
Gebru & Torres 2024, §3, citing Galton 1869
- 1883 Eugenics
Galton coins the word 'eugenics'
The project of improving the 'human stock': encourage the 'fit' to breed more (positive eugenics), prevent the 'unfit' from passing on their heredity (negative eugenics).
Gebru & Torres 2024, §3
- 1909 Eugenics
California begins forced sterilizations
State eugenics programs sterilize people deemed 'unfit'. California's program — which continued until 1979 — was later adopted by Nazi Germany as a template for 'racial hygiene' policies.
Gebru & Torres 2024, §3, citing Black 2003; Stern et al. 2017
- 1931 Eugenics
Eugenics as utopia
Many first-wave eugenicists framed their project in utopian terms — H.G. Wells among them. Galton himself had penned a utopian novel, The Eugenic College of Kantsaywhere, where reproduction required passing psychometric tests.
Gebru & Torres 2024, §4.2, citing Wells 1902, Wells et al. 1931, Sweet 2011
- 1957 Transhumanism
Julian Huxley names 'transhumanism'
Huxley — soon to be president of the British Eugenics Society (1959–62) — writes that by controlling 'the mechanisms of heredity', the human species can 'transcend itself … in its entirety, as humanity'.
Gebru & Torres 2024, §4.1, citing Huxley 1957
- 1955 The AGI race
The field of 'artificial intelligence' is launched
A workshop proposal aims at 'machines performing the most advanced human thought activities'. Grandiose promises later trigger the 'AI winters' — and researchers abandon the term for decades.
Gebru & Torres 2024, §1 and §5.1, citing McCarthy et al. 1955
- 1979 Eugenics
First-wave eugenics winds down — without vanishing
California's sterilization program ends. The British Eugenics Society persists under new names (the Galton Institute, then the Adelphi Genetics Forum). Criticism of eugenics grows through the 1970s.
Gebru & Torres 2024, §3
- 1988 Extropianism
The Extropy Institute is founded
Max More and Tom Morrow create the first organized group of modern transhumanists, committed to 'boundless expansion' and 'dynamic optimism'. A 'second wave' of eugenics takes shape around new genetic technologies: enhancement as personal choice.
Gebru & Torres 2024, §4.1, citing Regis 1994
- 1991 Singularitarianism
'Singularitarian' is coined
An Extropian named Mark Plus coins the word for believers in a coming technological Singularity — the moment self-improving AI outruns human history.
Gebru & Torres 2024, §4.1
- 1996 Extropianism
The Extropian mailing list shows its colors
Messages later surfaced by Émile Torres include leading figures ranking races by 'intelligence' — and members worrying the movement could be branded 'neo-eugenics, racist, neo-nazi' by the press.
Gebru & Torres 2024, §4.2, citing Torres 2023a
- 1998 Transhumanism
The World Transhumanist Association is founded
Nick Bostrom and David Pearce found a 'more mature and academically respectable form of transhumanism'.
Gebru & Torres 2024, §4.1, citing Bostrom 2005a
- 2000 Singularitarianism
The Singularity Institute is founded
Eliezer Yudkowsky founds the Singularity Institute for Artificial Intelligence — later renamed the Machine Intelligence Research Institute — with more than US$1.6 million from billionaire Peter Thiel.
Gebru & Torres 2024, §5.3
- 2003 Longtermism
Bostrom computes the far future
In 'Astronomical Waste', Bostrom estimates the accessible universe could host 10⁵⁸ digital people — making even minuscule reductions in 'existential risk' worth more than anything happening today. The seed of longtermism.
Gebru & Torres 2024, §4.1, citing Bostrom 2003, 2014
- 2005 Singularitarianism
The Singularity Is Near
Ray Kurzweil's bestseller predicts the Singularity for 2045: humans merge with machines, and our descendants 'wake up' the universe.
Gebru & Torres 2024, §4.1, citing Kurzweil 2005
- 2007 The AGI race
'Artificial general intelligence' gets its name
The first book on AGI, edited by Cassio Pennachin and Ben Goertzel. The term was suggested by Shane Legg, later a DeepMind co-founder; Goertzel's aim was 'transhuman AGI'. Contributors note 'general intelligence' was an unpopular, taboo-adjacent subject — controversies 'stoked by The Bell Curve'.
Gebru & Torres 2024, §5.1–5.2, citing Pennachin & Goertzel 2007b
- 2009 Rationalism
LessWrong launches
Yudkowsky founds the community blog 'dedicated to improving human reasoning and decision-making'. Rationalism becomes a movement — and superintelligent AI its central preoccupation.
Gebru & Torres 2024, §4.1
- 2010 Cosmism
A Cosmist Manifesto — and DeepMind
Goertzel publishes his manifesto for remaking the universe. The same year, Demis Hassabis, Mustafa Suleyman and Shane Legg found DeepMind, funded by Elon Musk, Peter Thiel and Jaan Tallinn — Thiel approached at his mansion after the Singularity Summit.
Gebru & Torres 2024, §4.1 and §5.3, citing Goertzel 2010; Shead 2017, 2020
- 2011 Effective Altruism
Effective Altruism organizes
The Centre for Effective Altruism forms in Oxford around William MacAskill and Toby Ord: do the most good possible, by explicit calculation. Its initial focus is global poverty.
Gebru & Torres 2024, §4.1
- 2014 The AGI race
Superintelligence — and Google buys DeepMind
Bostrom's book argues the 'default outcome' of misaligned AGI is 'doom' — while insisting an 'aligned' AGI should still be built. Musk and Thiel are both influenced; Musk calls AGI the 'biggest existential threat'.
Gebru & Torres 2024, §5.3, citing Bostrom 2014; Dowd 2017
- 2015 The AGI race
OpenAI is founded
Musk, Thiel, Sam Altman and others pledge US$1 billion to build AGI that 'benefits all of humanity'. Open Philanthropy — the Effective Altruist funder — adds a US$30 million grant.
Gebru & Torres 2024, §5.3, citing Novet 2015
- 2019 Longtermism
Longtermism is systematized
Hilary Greaves and William MacAskill publish 'The Case for Strong Longtermism': for evaluating actions, 'we may simply ignore all the effects contained in the first 100 (or even 1,000) years'. Effective Altruism's leadership pivots to the far future.
Gebru & Torres 2024, §4.1, citing Greaves & MacAskill 2019
- 2021 The AGI race
Anthropic is founded
Former OpenAI vice-presidents raise US$704 million within a year — most of it from figures aligned with the bundle: Jaan Tallinn, Dustin Moskovitz, and Sam Bankman-Fried's FTX (US$500 million).
Gebru & Torres 2024, §5.3, citing Coldewey 2022; Sambo et al. 2023
- 2022 The AGI race
ChatGPT — 100 million users in two months
OpenAI's chatbot becomes 'the greatest product launch in tech history'. Microsoft's investment grows to a reported US$10 billion the following year.
Gebru & Torres 2024, §5.3, citing Weil 2023; Bass 2023
- 2023 The AGI race
Pause letters, e/acc, and xAI
The Future of Life Institute's 'pause giant AI experiments' letter is signed by many of the same people racing to build AGI. A new variant — 'effective accelerationism' (e/acc) — declares any deceleration will 'cost lives'. Musk, three months after signing the pause letter, founds xAI.
Gebru & Torres 2024, §4.2 and §6.4
- 2024 The AGI race
The bundle gets a name
Timnit Gebru and Émile P. Torres publish 'The TESCREAL bundle' in First Monday, tracing the ideologies driving the race to build AGI back to first-wave eugenics — and urging a return to well-scoped, testable systems.
Gebru & Torres 2024
Who holds the threads
The same names, again and again
The bundle is not seven separate communities — it is largely the same people wearing different hats. Each ring below is one ideology; select a person to see their affiliations, in the paper's words, and where their money went.
Where the money went
- Peter Thiel US$1.6M+ Machine Intelligence Research Institute
Thiel funded Yudkowsky's Singularity Institute (later MIRI) with more than US$1.6 million. · Gebru & Torres 2024, §5.3
- Peter Thiel early funding DeepMind
Hassabis secured Thiel's investment after the 2010 Singularity Summit, at Thiel's California mansion. · Gebru & Torres 2024, §5.3, citing Shead 2020
- Elon Musk US$1B pledged OpenAI
Musk, Thiel, Altman and others collectively pledged US$1 billion at OpenAI's founding in 2015. · Gebru & Torres 2024, §5.3, citing Novet 2015
- Open Philanthropy US$30M OpenAI
Open Philanthropy's grant to OpenAI, 2017. · Gebru & Torres 2024, §5.3
- Sam Bankman-Fried US$500M Anthropic
FTX and Alameda Research invested US$500 million in Anthropic — most of its US$704M first-year raise came from figures aligned with the bundle. · Gebru & Torres 2024, §5.3, citing Coldewey 2022
- Dustin Moskovitz top contributor Machine Intelligence Research Institute
Moskovitz and Buterin are listed among MIRI's top contributors. · Gebru & Torres 2024, §5.3
- Jaan Tallinn early funding DeepMind
Tallinn was among DeepMind's early backers, with Musk and Thiel. · Gebru & Torres 2024, §5.3, citing Shead 2017
- Open Philanthropy US$5.16M xAI
Open Philanthropy awarded US$5,160,000 to the Center for AI Safety, whose executive director advises Musk's xAI. · Gebru & Torres 2024, §5.3
The bundle's theology
Two sides of the same coin
Every ideology in the bundle tells the same two stories about AGI. Slide between them.
Paradise
“The complete abolition of suffering in Homo sapiens … a cosmic rescue mission to promote paradise engineering throughout the universe.”
“We will transcend our frail bodies with all their limitations. Illness, as we know it, will be eradicated.”
“It is clear that we will have unlimited intelligence and energy before the decade is out.”
“Abundances of wealth, growth … to all minds who so desire.”
Doom
“The 'default outcome' of a misaligned AGI will be 'doom'.”
“The vast power of superintelligence could … lead to the disempowerment of humanity or even human extinction.”
“Some leading figures put the probability of an existential catastrophe this century at 16–20 percent or more; Eliezer Yudkowsky considers doom from near-term AGI more or less certain — and has endorsed military strikes on 'rogue' data centers, if necessary, to stop it.”
“Extinction is not something in the very distant future … And once we're all extinct, you know, all these other issues cease to even matter.”
Both stories assume the same machine-god. And both, Gebru & Torres argue, are used to justify building it — utopia as the prize, apocalypse as the reason only the 'right' people can be trusted with the work.
While the future is being argued about
Meanwhile, on Earth
The race toward the machine-god is not hypothetical. Its costs are being paid now — mostly by people far from the mansions where it is funded.
-
US$1.32/hour
The people behind the curtain
Building systems marketed as steps toward AGI depends on millions of workers labeling data. In 2023, Time reported that Kenyan workers were paid as little as US$1.32 an hour to label descriptions of sexual abuse, hate speech and violence for OpenAI's filters — work that left them, in their words, 'mentally scarred'.
Perrigo, Time (2023); Gebru & Torres 2024, §6.3
-
hundreds of billions of parameters
A planetary energy bill
Systems advertised as 'general' require models and datasets orders of magnitude larger than task-specific tools — with staggering environmental costs for training and inference, borne disproportionately by communities already facing the climate crisis.
Luccioni et al. (2023); Bender, Gebru et al. (2021); Gebru & Torres 2024, §6.2
-
5 billion images
Nobody checked what went in
At these scales, builders are less likely to curate or even understand their datasets. LAION-5B — used to train image generators with millions of daily users — was taken down in December 2023 after child sexual abuse material was found in it.
Thiel (2023); Birhane et al. (2021); Gebru & Torres 2024, §6.2
-
one CTO's story
Resources flow to the machine-god
The chief technology officer of Lesan AI — a startup doing machine translation for Ethiopian languages — reported investors walking away because they believed OpenAI and Meta had made his company obsolete. Evidence shows the big models perform poorly in exactly those languages.
Donastorg (2023); Hadgu et al. (2023); Gebru & Torres 2024, §6.2
-
3 days
A machine for everything can be tested for nothing
Meta's Galactica was advertised as able to 'summarize academic papers, solve math problems … and more'. Its public demo lasted three days before being pulled — users had produced authoritative-sounding nonsense on everything from crushed glass to antisemitism. A system with no defined purpose has no defined safety test.
Greene (2022); Khlaaf (2023); Gebru & Torres 2024, §6.1
-
'slightly conscious'
The machine takes the blame
Calling these systems 'thinking' or 'sentient' shifts attention from the corporations that build them. Chatbots have been deployed for mental-health advice without telling users, and lawyers have been fined for filing fake, chatbot-generated court cases. In one tragic case in Belgium, a man reportedly died by suicide after an AI chatbot encouraged him.
Biron (2023); Milmo & Agency (2023); Xiang (2023); Gebru & Torres 2024, §6.3
The language of 'safety' works twice, the paper argues: first to drive resources toward building AGI, then to steer scrutiny away from the harms of building it — toward a hypothetical apocalypse.
The alternative
Unbundling
Gebru & Torres end with an engineering argument, not a prophecy. Systems built for a defined job — detecting a crop disease, translating a specific language — can be specified, tested, and made safe. A machine built to do “anything” cannot. Their call: stop trying to build an all-knowing machine, and build well-scoped tools that serve the communities using them.
The bundle draws its power from being invisible — from making one very particular dream look like destiny. Now you can see the threads, and ask who is holding them.