🌍⚠️

Existential Risks

Disclaimer:
The topic of existential risks (or x-risks) is not a light-hearted one, and we recognize that delving into its implications can be emotionally challenging. Our primary objective is to foster planetary awareness and expand the emerging worldview of planetary stewardship, as we believe it can contribute to a brighter, safer, wiser and fairer future. To accomplish this, we must confront the dangers that could inhibit such a world from coming to fruition. Therefore, it is crucial that we diligently research and, collectively, address existential risks, employing every resource and tool at our disposal. This page represents our ongoing and modest visual attempt to investigate this crucial subject. Please note, Globaïa does not adhere to the 'TESCREAL bundle' of ideologies, as we align ourselves with a broader, more inclusive approach to planetary stewardship and existential risk management. Our commitment prioritizes ecological and social sustainability, integrating principles of indigeneity, planetary justice, and posthuman thought that emphasize respect for diverse cultural values, ecological wisdom, and the interconnectedness of human and non-human entities, challenging the techno-utopian visions often associated with TESCREALism.

Atomic Planet
Past nuclear tests are depicted with spikes, while a blue sphere symbolises the presence of a nuclear reactor.
The ‘ripple effect’ around reactors is showing the presence of antineutrinos produced as a byproduct of nuclear fission reactions that occur within these nuclear reactors.

Existential Risks refer to a threat that has the potential to either cause the extinction of humanity or irreversibly destroy our future potential for flourishing. These risks include events, processes, or phenomena that can severely undermine the long-term survival, well-being, or development of humanity on a global scale.

The profound and consequential nature of an existential catastrophe can be illustrated by a compelling example conceived by Derek Parfit in Reasons and Persons (1986), as cited by Toby Ord in The Precipice (2020):

Picture a catastrophic nuclear conflict that annihilates 99 percent of the global population. This devastating event would plunge the world into a centuries-long dark age before the survivors could eventually reconstruct civilization, returning it to its prior glory—battered, marked, but unbroken.

Now contrast this scenario with a war that wipes out the entire human population. Although the second conflict would undoubtedly be more disastrous, just how much more severe would it be? Either calamity would be unparalleled in history, resulting in billions of casualties. The second war would claim tens of millions of additional lives, making it worse on this basis. However, there is another, far more crucial distinction between the two conflicts.

While both wars claim billions of lives, the second one eradicates humanity entirely.
Both conflicts devastate our present, but the second one demolishes our future as well.

This qualitative distinction in the losses sustained during that final percentage is what sets existential catastrophes apart, making the reduction of existential risk an exceptionally critical priority.

Artificial Intelligence-Created Representation of Diverse Existential Threats
We are not blind to the ironic nature of employing an AI-produced illustration to explore existential risks, including AI itself as one such risk. This is a continuing experiment, and we welcome your thoughts and feedback.

These images embody our shared perception of how these events might transpire, serving as a testament to the power of our collective imagination. However, it is crucial to remember that the actual unfolding of these events, if they were to occur, would likely look and be experienced quite differently. Nevertheless, these visual representations provide an opportunity for us to confront our fears and aspirations, ultimately reminding us that it is within our power to prevent these events from happening. By acknowledging our role in shaping the future, we can work together to create a world where such events remain exclusively within the realm of imagination.

 

Existential Moods

Émile P. Torres, a thinker in the field of existential risk and philosophy, has conceptualised a framework delineating the evolution of Western thought regarding human extinction. This framework is structured around five distinct "existential moods", each representing a unique period in history characterised by specific beliefs and attitudes towards the possibility and nature of human extinction. Torres' analysis delves into the shifting perspectives from ancient times to the present day, exploring how scientific, philosophical, and cultural developments have dramatically altered our understanding of humanity's place in the universe and the threats to our continued existence.

The Five Existential Moods.
Read the book or a summary of it.

1. Eternalist Era (Ancient times to 1850s)

During this era, the dominant belief was that humanity could not be extinguished. This conviction was largely rooted in religious doctrines and the idea of a cosmic order, which assured people that a divine entity or the natural laws of the universe would perpetually safeguard humanity.

The emergence of secular thinking and scientific rationalism began to challenge the religious and mystical views of the world. This shift was marked by the Enlightenment and the gradual rise of scientific methods, which questioned the previously unchallenged religious doctrines about human permanence.

2. Cosmic Fragility (1850s to 1950s)

This period saw a dramatic shift to the acknowledgment of humanity's vulnerability and the eventual certainty of extinction. Influenced by scientific advancements, this mood was characterized by a sense of existential despair and the recognition that, like all species, humans are subject to the laws of nature and could face eventual oblivion.

Key scientific developments triggered this shift. The Second Law of Thermodynamics introduced the concept of entropy, suggesting a universe tending towards disorder and eventual 'heat death'. Darwin's theory of evolution further eroded the notion of human exceptionalism by placing humans within the continuum of natural species, all susceptible to extinction.

3. Brink of Oblivion (1950s to 1980s)

The focus shifted to the frightening realisation that humanity now possessed the technological means for its own rapid extinction, primarily through nuclear weapons. This mood was marked by a palpable fear of nuclear war and the potential for a self-inflicted end to human civilisation.

The development and deployment of nuclear weapons, particularly the atomic bombings of Hiroshima and Nagasaki, vividly demonstrated humanity's newfound destructive capabilities. The subsequent development and testing of even more powerful thermonuclear weapons (such as Castle Bravo, in 1954) solidified this mood, highlighting the immediate and tangible possibility of human-induced annihilation.

4. Unforeseen Hazard (1980s to 2000s)

A new realisation emerged that nature, previously considered a benign or even protective force for humanity, could actually pose severe existential risks. This mood was characterised by the understanding that catastrophic natural events could threaten human survival on a global scale.

A series of groundbreaking discoveries in earth sciences played a crucial role. The hypothesis and subsequent confirmation of an asteroid impact causing the dinosaur extinction shattered the prevailing uniformitarian view in geology, which had held that catastrophic events did not occur on a global scale. This revelation opened eyes to the fact that Earth is not exempt from such cataclysmic events.

5. Age of Polycrisis (2000s to present)

This current mood is dominated by a heightened awareness and concern over a diverse array of existential threats facing humanity in the XXIst century. These include climate change, technological advancements with unpredictable consequences, biodiversity loss, and the potential for a mass extinction event.

Two main developments triggered this shift. Firstly, the ethical reevaluation of human extinction, which framed such an event as a moral catastrophe of cosmic proportions. Secondly, the growing body of research highlighting the immediacy and severity of threats like climate change and biodiversity loss, coupled with the rapid development of potentially dangerous technologies such as artificial intelligence and genetic engineering, painted a stark picture of a future fraught with unprecedented risks.

 

X-Risks

These risks can arise from various sources, including natural events, human activities, and technological advancements. Prominent examples of natural existential risks include:

  • X-class flares pose an existential threat to humanity, as their massive solar emissions have the potential to wreak havoc on our planet's technological infrastructure and natural environment. These powerful solar storms could disrupt satellite communication, damage power grids, and trigger widespread blackouts, thereby crippling our modern way of life. Moreover, increased exposure to solar radiation may have dire consequences for the Earth's climate and ecosystems, exacerbating global warming and other environmental challenges. Historically, the Carrington Event of 1859 serves as a prime example of an extreme solar storm that caused widespread telegraph disruptions. More recently, the 1989 geomagnetic storm led to the collapse of Canada's Hydro-Québec power grid, leaving millions without electricity. These past events underscore the potential for X-class flares to cause unprecedented devastation on a global scale, potentially threatening the very existence of human civilisation.

  • Large asteroid collisions, involving objects 5 km or larger, occur approximately every twenty million years and can release energy a hundred thousand times greater than the largest bomb ever detonated. Land impacts could destroy areas comparable to a small country, while larger asteroids may lead to extinction-level events. Asteroid impacts are among the most well-understood risks in this context.

    There has been talk of methods to deflect asteroids on a collision course with Earth. The primary damage from an impact wouldn't be the initial collision, but rather the dust clouds projected into the upper atmosphere. An "impact winter" could alter the climate, harm the biosphere, disrupt food supplies, and cause political instability.

  • A supervolcano can produce eruptions with ejecta volumes exceeding 1,000 km3, thousands of times larger than typical eruptions. The primary threat is the aerosols and dust projected into the upper atmosphere, which would absorb sunlight and trigger a global volcanic winter. The 1991 Mt. Pinatubo eruption caused a 0.5°C global cooling for three years, while the Toba eruption around 70,000 years ago may have cooled the planet for over two centuries. The effects of these eruptions are comparable to a nuclear war, with more violent eruptions but fewer secondary effects like firestorms.

  • Cosmic risks, while relatively rare, pose significant threats to the existence of humanity, with the potential to cause catastrophic events that could lead to our extinction. Among these cosmic phenomena, gamma-ray bursts, nearby supernovae, and black hole mergers stand as prominent harbingers of potential doom. Gamma-ray bursts, the most energetic explosions in the universe, can release a torrent of lethal radiation that could annihilate Earth's protective atmosphere and trigger mass extinctions. Similarly, a nearby supernova, which occurs when a massive star exhausts its nuclear fuel and explodes, could shower our planet with harmful cosmic radiation, causing long-term climatic changes and threatening life as we know it. Lastly, black hole mergers, the cataclysmic collision of two black holes, generate powerful gravitational waves and release colossal amounts of energy, which, if close enough, could have disastrous consequences for our solar system and ultimately, life on Earth.

  • An epidemic of infectious disease can spread across large regions or even globally, with the likelihood of a high-impact epidemic potentially higher than assumed. Nature already contains devastating disease features such as incurability (Ebola), high fatality (rabies), extreme infectiousness (common cold), and long incubation periods (HIV). A pathogen combining these traits could cause extreme death tolls. Although modern sanitation, medical research, and disease-fighting institutions exist, rapid transportation, dense populations, and slums can still facilitate the rapid spread of diseases. The COVID-19 pandemic, a stark reminder of our vulnerability, has led to millions of deaths, exposed weaknesses in healthcare systems, and disrupted economies worldwide, emphasizing the urgent need for global preparedness against future pandemics.

Human activities and land conversion can exacerbate global pandemics, which emerge at the intersection of nature and culture. Conversely, the prominent existential risks that are anthropogenic, stemming from human actions, include:

  • Uncertainties in risk estimates suggest that extreme warming and feedback loops could raise global temperatures by 4°C or 6°C above pre-industrial levels. Factors like methane release from permafrost or Amazon rainforest dieback may contribute. Poorer countries, most affected, could become uninhabitable, causing mass deaths, famines, social collapse, and mass migration. Agricultural and biosphere shocks in developed countries could lead to global conflict and potential civilization collapse. Past civilisation collapses driven by climate change support this risk.

  • An ecosystem experiencing a significant, potentially irreversible decrease in carrying capacity for all organisms may lead to mass extinction. As part of the global ecosystem, humans fundamentally rely on it. Current species extinction rates far exceed historical norms, and humanity operates well beyond a safe ecological range. The compounding effects of ecological degradation put the entire world at risk. Although some human lifestyles could potentially be sustained independently from ecosystems, doing so on a large scale poses technological and ethical challenges.

  • Synthetic biology involves designing and creating biological devices and systems for practical applications, which introduces human intentionality to traditional pandemic risks. Current regulation efforts are nascent and may lag behind research advancements. One of the most harmful consequences of synthetic biology could stem from an engineered pathogen targeting humans or essential ecosystem components, originating from military or commercial bio-warfare, bio-terrorism, or lab leaks. The integration of synthetic biology products into the global economy or biosphere may create additional vulnerabilities, such as widespread benign products being targeted to cause damage.

  • Atomically precise manufacturing enables high-throughput processes at the atomic or molecular level, creating innovative products like smart or highly resilient materials. This could empower various groups or individuals to produce a range of items, potentially including large weapon arsenals or even nuclear bombs. However, nanotechnology could also address global issues like resource depletion, pollution, climate change, clean water, and poverty. Risks include self-replicating nanomachines consuming the environment or the misuse of medical nanotechnology.

  • AGI, or Artificial General Intelligence, refers to machine or software intelligence that achieves human-level cognitive abilities across various domains. It involves the development of intelligent agents that perceive their environment and act to maximize success. Highly advanced AGI may be challenging to control and could endeavor to enhance its own intelligence and acquire resources, potentially disregarding human survival and values. This makes AGI a unique risk, with extinction being more probable than lesser impacts. However, such intelligence could also combat other risks, offering great potential. AGI-driven warfare and related technologies pose additional risks. "Whole brain emulations," where human brains are scanned and replicated in machines, could potentially alleviate some concerns.

  • The probability of a full-scale nuclear war between the USA and Russia has increased dramatically in 2022, with rhetoric reminiscent of the past century resurfacing. The potential for intentional or accidental conflict is real, with some estimates placing the risk at 10% over the next century. A larger impact would hinge on whether the war induced a nuclear winter, causing global freezing temperatures and potential ozone layer destruction. Firestorms in targeted cities would need to lift soot into the stratosphere. Recent models confirm severe risks, with global food supply disruption leading to mass starvation and state collapse.

  • Global economic or societal collapse encompasses a wide range of conditions, often involving economic downturns, social chaos, civil unrest, and even the disintegration of human societies and life support systems. The complex, interconnected nature of the global economic and political system renders it vulnerable to unexpected system-wide failures, even if individual components are reliable. This interconnectedness creates systemic risk, as self-reinforcing joint risks can spread throughout the system and potentially impact related external systems. Such effects have been observed in ecology, finance, and critical infrastructure like power grids. The risk of collapse increases when multiple independent networks rely on each other.

  • Governance disasters fall into two categories: failing to address solvable problems and actively causing worse outcomes. Examples include not alleviating absolute poverty and establishing a global totalitarian state. Technological, political, and social changes could create new governance forms, with varying outcomes. The challenges with governance disasters include estimating their likelihood and determining their impact, as it's unclear how to objectively compare continued poverty, global totalitarianism, and catastrophic events resulting in billions of casualties or civilization collapse.

Adapted and extended from: Pamlin, D. & Armstrong, S. (2015) Global Challenges · 12 Risks that threaten human civilisation, Global Challenges Foundation & Future of Humanity Institute, Oxford University.

Global catastrophic challenges may also involve Unknown Unknowns, which are composed of seemingly unrelated risks that, when combined, create a significant danger, frequently referred to as a Polycrisis. The Fermi Paradox, emphasising the absence of detectable extraterrestrial life, could suggest that intelligent life commonly self-destructs before spreading across the galaxy, an idea known as the Great Filter and captured by the final variable (L) in the famous Drake Equation.

Cosmic Significance
If humanity is the sole moral agent in the universe, our responsibility is immense, as we hold the potential for all moral action and the ability to shape the universe in a just and ethical manner.
If we are the only beings capable of contemplating the universe, it is our duty to seek understanding and foster the process of the universe comprehending itself.
And if Earth is the unique origin of life, our role as stewards of life becomes even more crucial, as we are responsible for preserving, nurturing, and potentially spreading life throughout the cosmos.
Therefore, it could be argued that humanity’s extinction would be an event of cosmic significance.

Since the 🌍⏳ Anthropocene started, our awareness of existential risks has grown significantly as scientific advancements, global communication, and public discourse have facilitated a deeper understanding of the potential threats facing humanity. The advent of the atomic bomb in the 1940s marked a pivotal moment in recognizing our capacity to cause our own extinction. This realization sparked a global movement for nuclear disarmament and arms control, leading to international treaties and ongoing diplomatic efforts.

As we advanced into the latter half of the 20th century, growing concerns about environmental degradation and pollution culminated in the recognition of anthropogenic climate change as a major existential threat. The establishment of the Intergovernmental Panel on Climate Change (IPCC) in 1988 reflected the rising urgency to address this issue. Furthermore, advances in technology and artificial intelligence have raised ethical and safety concerns, leading to organisations such as the Future of Life Institute, the Centre for the Study of Existential Risk and the Global Challenges Foundation promoting responsible development and research.

The interconnectedness of our global society has heightened awareness of the risks associated with pandemics, as evidenced by the widespread impact of HIV/AIDS, SARS, and most recently, COVID-19. Finally, Russia's invasion of Ukraine has revived the nuclear discourse and risk that was once believed to be a relic of the Cold War era.

 

🌍 Planetary Fever 🌡️

High temperatures have been linked to increased mortality, decreased labour productivity, decreased cognitive performance, impaired learning, adverse pregnancy outcomes, decreased crop yield potential, increased conflict, hate speech, migration and infectious disease spread.

Climate change has been a topic of discussion and concern for many years, with increasing evidence pointing towards its potentially catastrophic impact on the future of our planet. But beyond the direct effects on the natural world, climate change also poses a significant existential threat to human civilisations. By examining the history of past civilisations and the interconnectedness of our modern world, we can understand why climate change has the potential to bring about widespread destruction and destabilisation.

Past societies have faced and been impacted by environmental changes, such as droughts, floods, and temperature shifts. In some cases, these changes were so severe that they contributed to the collapse of entire civilisations. This serves as a cautionary tale for our modern world, as the interconnectedness of our global systems makes us even more vulnerable to the effects of climate change.

Read more on our dedicated pages on 🌏⚖️ Planetary Tipping Systems and the 🌎🚨 Planetary Emergency.

The migration towards the poles of Human Climate Niches according to various global annual average temperature scenarios by 2090-2100 (current trend is +2.7 °C) will result in the expansion of areas where people will be dangerously exposed to unprecedented heat (>29 °C). At +2.7 °C, two billion people will be exposed to dangerous heat (22% of the projected end-of-century population). In worst-case scenarios of +3.6 °C or even +4.4 °C global warming, half of the world’s population could be left outside the climate niche.
Source: Lenton, T.M., Xu, C., Abrams, J.F. et al. Quantifying the human cost of global warming.Nat Sustain (2023). https://doi.org/10.1038/s41893-023-01132-6

To learn more about theses visualisations and watch the short film,
visit our page on 🌍🌡️ Human Climate Niches.

Projected State of Uninhabitability by 2070
To learn more about this visualisation and watch the short film, visit our page on 🌍🌡️ Human Climate Niche.

Planetary Riskscape
The stability of the Holocene era has eluded Earth, leaving us to traverse an unpredictable ocean with restricted foresight. However, science can offer glimpses into the path that lies ahead. To circumvent a perilous and inequitable future, immense endeavors and cooperation on a global level are necessary to overcome the challenges presented by the Anthropocene epoch. Our deficiency lies not in knowledge but in the unity of our collective strength.
We created this image for the book Breaking Boundaries: The Science of Our Planet by Johan Rockström and Owen Gaffney.

 

⚛️ Nuclear Perils ☢️

The worldwide significance of nuclear risks, both civil and military, cannot be overstated, as the potential consequences of nuclear accidents or warfare could be catastrophic for humanity and the environment.

On the civil side, incidents such as the Chernobyl disaster in 1986 and the Fukushima Daiichi nuclear disaster in 2011 have demonstrated the far-reaching implications of nuclear accidents, including widespread radioactive contamination, long-term health effects, and significant economic burdens. These incidents highlight the importance of strict safety regulations and risk management for nuclear facilities.

In the military domain, the destructive power of nuclear weapons poses an existential threat to global security. The bombings of Hiroshima and Nagasaki in 1945 exemplify the devastating consequences of nuclear warfare, leading to immense loss of life and long-lasting environmental damage.

A Riddled Planet
During WWII, the Allied forces conducted widespread bombing operations in the Pacific. The two long spikes in Japan represent Hiroshima and Nagasaki.
It is easy to see things in a cold light from this distance high above Earth and decades after the events. But take a moment to consider the amount of suffering that each of these dots caused.
The height of the spikes illustrates the energy unleashed by these bombs, highlighting the unparalleled nature of nuclear weapons that marked the beginning of a new era in existential threats.
Data: Theater History of Operations (THOR) Data: World War II

The Doomsday Clock, a symbolic representation of the likelihood of global catastrophe, was created in 1947 by the Bulletin of the Atomic Scientists. This ominous timepiece reflects the potential for humanity to self-destruct through a combination of nuclear war, climate change, and emerging technologies. By measuring the "minutes to midnight," the clock seeks to raise awareness and inspire action to prevent global disaster. Throughout its history, the clock's hands have been adjusted in response to geopolitical events and scientific developments, making it a stark reminder of the ever-present threats that imperil our planet and the urgent need for global cooperation to address these challenges.

The Doomsday Clock, 1947-2024
Since its inception in 1947, the Doomsday Clock has served as a symbolic representation of humanity's proximity to global catastrophe, primarily driven by nuclear threats and, more recently, climate change and technological risks. Initially set at seven minutes to midnight, the clock's minute hand has shifted over the years in response to significant geopolitical and scientific events. Notable moments include the clock's earliest setting of two minutes to midnight in 1953, following the development of the hydrogen bomb, and the furthest setting of 17 minutes to midnight in 1991, after the end of the Cold War and the signing of the Strategic Arms Reduction Treaty. In recent years, the clock has steadily crept closer to midnight, reflecting growing concerns over climate change, nuclear proliferation, and emerging technologies. As of 2024, the clock stands at an alarming 90 seconds to midnight, highlighting the urgent need for collaborative global action to avert potential catastrophes.

Cesium-137 deposition following the nuclear disasters at Fukushima and Chernobyl has had a significant impact on the environment and public health. The Chernobyl accident in 1986 led to widespread deposition of radioactive materials across Europe, with Cs-137 being one of the most persistent radionuclides. Similarly, the Fukushima Daiichi nuclear power plant disaster in 2011 resulted in the release of vast amounts of radioactive isotopes, including Cs-137, which contaminated land, air, and water. In both cases, Cs-137 deposition has led to long-term contamination of ecosystems, as the isotope's half-life of 30 years allows it to persist in the environment. This has resulted in the exclusion of vast areas surrounding the disaster sites, with human resettlement deemed unsafe due to elevated radiation levels.

Cesium-137 deposition after the Fukushima (2011) and Chernobyl (1986) disasters.
Data: Christoudias, T. and Lelieveld, J.: Modelling the global atmospheric transport and deposition of radionuclides from the Fukushima Dai-ichi nuclear accident, Atmos. Chem. Phys., 13, 1425–1438, 2013
UNSCEAR Scientific Committee on the Effects of Atomic Radiation, Assessments of the radiation effects from the Chernobyl nuclear reactor accident.

Nuclear tests, conducted primarily during the mid-20th century, were a means for nations to develop and demonstrate their nuclear capabilities. The first recorded nuclear test took place on July 16, 1945, when the United States tested the "Trinity" device in New Mexico. The period of intensive nuclear testing spanned from the 1940s to the early 1990s, with the peak occurring during the Cold War era between the United States and the Soviet Union. Over 2,000 nuclear tests were conducted worldwide during this time, with the United States, the Soviet Union, the United Kingdom, France, and China being the primary participants. The Comprehensive Nuclear-Test-Ban Treaty (CTBT), adopted in 1996, has largely put an end to nuclear testing, although it has yet to enter into force due to the non-ratification by a few countries.

Today, there are approximately 440 operational nuclear reactors worldwide, with the majority being utilised for civilian purposes, such as electricity generation. These civil nuclear reactors are regulated and monitored to ensure their safe operation and prevent the proliferation of nuclear weapons. On the other hand, military nuclear threats primarily stem from countries possessing nuclear weapons or seeking to develop them, as well as the risk of nuclear materials falling into the hands of non-state actors, such as terrorist organizations.

Past Nuclear Detonations (🔴 red dots) and Nuclear Reactors (🔵 blue dots).
The pair of red dots in Japan represent the bombings of Hiroshima and Nagasaki during World War II, which marked the sole instances of nuclear weapons being employed against civilian populations. These events led to immense suffering and devastation on an unprecedented scale. Another notable event is the detonation of the Tsar Bomba in 1961 in Novaya Zemlya. This nuclear weapon, developed by the Soviet Union, remains the most powerful device ever detonated by humanity. With a yield of 50 megatons, the explosion's force was equivalent to 3,800 times the power of the atomic bomb dropped on Hiroshima. The shockwave created by the Tsar Bomba was so powerful that it circled the Earth three times before dissipating.
Data: IAEA Power Reactor Information System | Usman, S.M. et al. AGM2015: Antineutrino Global Map 2015. Sci. Rep. 5, 13945; doi: 10.1038/srep13945 (2015) | Johnston Archive of Nuclear Tests

 

👾 Digital Minds 🤖

The advancement of Artificial Intelligence (AI) could eventually pave the way for the development of Artificial General Intelligence (AGI) — a term sometimes used interchangeably with God-like AI (GAI). This progression could then lead to the emergence of Artificial Superintelligence (ASI), presenting a spectrum of existential threats to humanity.

A useful definition for AGI or Powerful AI is proposed by Dario Amodei, CEO of Anthropic:

A scalable, multi-instance AI surpassing human experts across disciplines, autonomously executing complex tasks via virtual interfaces and remote control, functioning as a “country of geniuses” within a datacenter.

For a deeper exploration, see his essay Machines of Loving Grace.

As AI systems evolve to become more sophisticated and autonomous, the likelihood of unintended outcomes and human oversight diminishes. This is especially true with AGI, which would possess human-like cognitive abilities and could surpass human performance in nearly any task.

“A superintelligent AI system that is autonomous and goal-directed would be a potentially rogue AI if its goals do not strictly include the well-being of humanity and the biosphere, i.e., if it is not sufficiently aligned with human rights and values to guarantee acting in ways that avoid harm to humanity.”

Further explore this argument by reading “How Rogue AIs may Arise” authored by Prof. Yoshua Bengio.

The pathway from Artificial Intelligence (AI) to Artificial General Intelligence (AGI), further to Artificial Super Intelligence (ASI), and finally to potentially Rogue Artificial Intelligence (RAI) isn't necessarily a clear-cut or inevitable one. However, there does appear to be potential for this progression if the objectives of the AI system do not consistently align with human values across various cultures and periods.

This is why, considering the abilities of current large language models (LLMs) as of 2023, the potential extinction-level threats posed by AI have become increasingly prominent in public dialogues and policy-making forums. This has led to near-consensus statements like:

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

— Statement on AI risks by leading technology experts (May 2023)

Modern Technosphere
The global technosphere is a vast and interconnected network of physical infrastructure, digital systems, and devices that facilitate the exchange of information and enable the functioning of modern technologies. This intricate web of components includes transmission lines, submarine cables, cloud centres, and data centres, which play a crucial role in supporting the Internet and the training of machine learning models that could potentially lead to artificial general intelligence (AGI).

Global map of data centers, revealing a new layer of the Technosphere: we call it the Xenoösphere. The term derives from two Greek roots: xénos (ξένος), meaning “stranger” or “alien,” and noûs (νοῦς), meaning “mind” or “intellect.” Together, the name signifies “the sphere of alien minds” or “the domain of emergent, non-human intellect.”

This emerging sphere is shaped by a set of key players, not quite Mad Max but MAAMAX—an acronym for the corporations (represented by colored circles) driving the current AI boom through the training of frontier large language models, at least in the Western world. Microsoft (supporting OpenAI and Anthropic), Amazon (backing Anthropic), Alphabet (through DeepMind), Meta, Apple, and xAI are collectively forming the foundation of this new technological epoch.

Data kindly provided by DataCenterMap.com

Over recent years, the realm of machine learning has seen remarkable progress, epitomised by state-of-the-art models like GPT-4. These models exhibit superior proficiency in natural language comprehension and generation, signaling a new era in the field. This upswing, a chapter in the global narrative of the 🌍🎢 Great Acceleration, is underpinned by three key factors: (1) burgeoning model sizes, (2) the advent of more sophisticated architectures, and (3) advanced training algorithms. These enhancements have ushered in superior generalisation abilities. A prevailing sentiment among many specialists is that relentless refinement and advancement of these models may eventually yield AGI capable of performing any cognitive task at a human level. Yet, the unanswered, vital question that looms for researchers and policymakers alike is whether such formidable AGI systems will align with human values and aims, ensuring the birth of Friendly AI (FAI).

Navigating the philosophical labyrinth of AI alignment is as demanding as its technical and practical counterparts. It resembles an 'Axiological Singularity'—a concept signifying the immense intricacy and seemingly insurmountable hurdle of imbibing human value systems into an artificial sentient entity. The task's complexity is compounded by the flux and diversity innate to human values. Ironically, humans themselves, the source of these values, struggle to consistently comprehend and adhere to their own fluctuating value systems—systems that are a product of time and cultural divergence.

Three Eras of Machine Learning
1. The Pre-Deep Learning Era: In the period preceding the advent of Deep Learning, the growth of training computation power largely adhered to Moore's Law, which predicted a doubling of computational capacity approximately every 20 months.
2. The Deep Learning Era: Between 2010 and 2012, the Deep Learning revolution began, exhibiting a significantly faster growth rate in training compute, with a doubling time of around 6 months.
3. The Large-Scale Era: Starting between 2015 and 2016, a distinct trend emerged, characterised by models developed by major corporations that employed training compute on a scale 2-3 orders of magnitude larger than systems adhering to the Deep Learning Era trend within the same timeframe. Interestingly, the growth rate of compute for these Large-Scale models appears to be slower, with a doubling time of approximately 10 months.
Data: Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn, Pablo Villalobos, Compute Trends Across Three Eras of Machine Learning. 2022. arXiv:2202.05924
Figure: Adapted from: Sastry, G., Heim, L., Belfield, H., Anderljung, M., Brundage, M., Hazell, J., O'Keefe, C., Hadfield, G. K., Ngo, R., Pilz, K., Gor, G., Bluemke, E., Shoker, S., Egan, J., Trager, R. F., Avin, S., Weller, A., Bengio, Y., & Coyle, D. Computing power and the governance of artificial intelligence. 2024. arxiv.org/2402.08797

AI technologies are quickly evolving, advancing in capabilities to produce text, images, and videos almost indistinguishable from those created by humans. While these advancements offer numerous advantages, they also bring forth potential risks. AI could be utilised to spread bias, power self-operating weapons, foster the spread of false information, or facilitate cyberattacks. Moreover, while human supervision often accompanies the use of these AI systems, they are progressively gaining the ability to act independently, potentially leading to harmful outcomes.

As AI continues to progress, it may eventually present catastrophic or even existential threats. There exist a multitude of manners in which AI systems could instigate or augment large-scale risks, a few of which are outlined below.

Adapted from Dan Hendrycks & Mantas Mazeika, X-Risk Analysis for AI Research, 2022, https://arxiv.org/abs/2206.05862 & Center for AI Safety (CAIS), https://www.safe.ai/ai-risk

You can examine this chart featuring 24 scenarios illustrating the various ways humanity and AGI/ASI may coexist, with outcomes ranging from positive to absolutely negative. This exploratory exercise is inspired by Max Tegmark's Life 3.0 and Nick Bostrom's Superintelligence: Paths, Dangers, Strategies.

24 Scenarios for Artificial Superintelligence and Humans
Approach these scenarios cautiously. Although many of these scenarios seem positive, researchers such as Eliezer Yudkowsky caution that without thorough preparation and precision, the emergence of AGI could likely result in humanity's extinction. It goes without saying that certain scenarios can be quite speculative, such as the Cosmic Coalition.

Here, we present in a visual form 9 scenarios of an AGI takeover, each warranting cautious consideration. An abundant body of literature exists online detailing the nature, paths, kinetics, and problems to be addressed in order to avert these potential outcomes. However, it is crucial to apply the precautionary principle to this invention, possibly more so than any other development in human history.

Artificial General Intelligence (AGI) Takeover Scenarios
In a scenario reminiscent of various science fiction works, an artificial general intelligence could potentially surpass human intellect and control, leading to a world dominated by a superintelligent entity with unforeseen consequences for humanity.
Sources & Inspirations:
· Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies, 2014.
· Yudkowsky, Eliezer. Artificial Intelligence as a Positive and Negative Factor in Global Risk, 2008.
· Future of Life Institute, Benefits & Risks of Artificial Intelligence, 2015.
· Brundage et al. The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation, 2018.
· Russell, Stuart. Human Compatible: Artificial Intelligence and the Problem of Control, 2019.
· Tegmark, Max. Life 3.0: Being Human in the Age of Artificial Intelligence, 2017.
· UNICRI, UNCCT & UNOCT. Algorithms and Terrorism: The Malicious Use of Artificial Intelligence for Terrorist Purposes, 2021.
· Yudkowsky, Eliezer. AGI Ruin: A List of Lethalities, 2022.

Corporations such as Nvidia, Intel, Samsung and Taiwan Semiconductor Manufacturing Company (TSMC) are leading the advanced microprocessor production space. They have been relentlessly pushing the boundaries of innovation by developing increasingly sophisticated microprocessors. These cutting-edge chips are manufactured using extreme ultraviolet (EUV) lithography machines exclusively designed and built by ASML, a Dutch multinational corporation and global leader in photolithography equipment. As the physical foundation for AGI continues to evolve, the powerful synergy between these industry giants is poised to play a pivotal role in realising its immense potential and risks.

Microprocessor Fab Labs (blue spikes)
A semiconductor fab, or fab lab, is a crucial component in the production of integrated circuits, which are created by transforming raw silicon wafers within these specialized manufacturing plants. These integrated circuits serve as the foundation for advanced technologies, including AI machine learning models. As the complexity and capabilities of AI models expand, there is a growing demand for more powerful and efficient integrated circuits.
Data: Frost, K. D.; Hua, I. (2019). Global Semiconductor Manufacturing Water Withdrawals. Purdue University Research Repository. doi:10.4231/NYGF-BH18

The immense scale of the AI boom, driven by cloud computing infrastructures and data centers powering the training of frontier AI models from companies like OpenAI, Anthropic, and DeepMind. Each blue dot represents a data center, the critical hubs of this technological revolution. The visualization intricately maps the transmission lines and submarine cables that form the backbone of the technosphere, illustrating the communication networks enabling the Internet. These networks carry and store the vast repository of humanity’s collective output—texts, videos, and sounds—used to train multimodal large language models, showcasing the interconnected systems that underpin the rise of artificial intelligence.

However, this rapid expansion comes at a significant energy cost. As of late 2024, data centers worldwide consume approximately 460 terawatt-hours (TWh) of electricity annually, accounting for about 1.4-1.7% of global electricity use. The International Energy Agency (IEA) projects that this consumption could double by 2026, potentially exceeding 1,000 TWh, comparable to the total electricity usage of Japan. In the United States, data centers’ share of electricity demand is expected to rise from 4% in 2022 to 6% by 2026.

Data kindly shared by DataCenterMap.com

If we, as a global community, decide to push the research agenda and help create a new form of intelligence on Earth, it must be done with extreme caution and vigilance. To ensure the safe and responsible advancement of AGI, it is crucial to involve international supervision from a recognised body, such as the United Nations, and adopt a collaborative model similar to CERN or ITER. Participating nations can pool resources, expertise, and knowledge while implementing strict security measures and conducting comprehensive research into AI alignment. Establishing the AGI research facility in a remote location, like the Kerguelen Islands, further mitigates potential threats and encourages a focused, secure environment for researchers.

The following is a collection of AI-generated images depicting fictional research facilities on the Kerguelen Islands. This playful experiment serves to demonstrate the remarkable capabilities of artificial intelligence in the realm of image creation, alongside the numerous other domains being disrupted by AI technology.

K-Island Project

K-ISLAND · Kerguelen Intelligence Systems Laboratory for Artificial Neural Development
A sequence of AI-produced images serves as an introspective exploration of the technology's disruptive potential and showcases its innovative prowess.

 

😷 Epidemics & Pandemics 🦠

The COVID-19 pandemic has had a profound worldwide significance, laying bare the vulnerabilities of modern civilization and exposing the interconnected nature of our global society. As the virus spread rapidly across the globe, overwhelming healthcare systems, disrupting economies, and claiming millions of lives, it became evident that even the most advanced nations were ill-prepared for a crisis of this magnitude. The pandemic has highlighted our dependence on global supply chains, the importance of timely and accurate information sharing, and the need for international cooperation to tackle common challenges.

Pandemonium
The spread of COVID-19 occurred rapidly during the first few months of the pandemic in 2020. Blue spikes correspond to cases while red ones correspond to deaths.
Data: Johns Hopkins University COVID-19 Tracker

The Ebola outbreak of 2014-2016 in West Africa was one of the most devastating epidemics in recent history, causing widespread fear and panic across the globe. The highly contagious and deadly Ebola virus, transmitted through contact with bodily fluids, ravaged communities and overburdened healthcare systems in the hardest-hit countries of Guinea, Sierra Leone, and Liberia. According to the World Health Organization, the outbreak resulted in over 28,600 cases and claimed more than 11,300 lives, marking it as the largest and deadliest Ebola outbreak on record. The gruesome symptoms of this terrible virus have marked people's imagination, serving as a stark reminder of the potential devastation that pandemics can bring.

Ebola
The probability of Ebola virus transmission in Tropical Africa, as determined by population-based modeling.
Source: Pigott et al. (2014) Mapping the zoonotic niche of Ebola virus disease in Africa eLife 3:e04395.

While the origins and nature of future pandemics remain uncertain, certain locations may be more susceptible than others and warrant vigilant observation.

Bushmeat markets, where wild animals are sold for human consumption, can serve as a breeding ground for zoonotic diseases, as close contact between humans, live animals, and their carcasses can enable the transmission of pathogens across species. Wet markets, which feature a variety of live animals in close quarters, pose similar risks, as they create an environment conducive to the exchange of pathogens between species, increasing the likelihood of disease spillover into human populations. Additionally, biohazard level 3 and 4 laboratories, which handle highly infectious and dangerous pathogens for research purposes, carry inherent risks of accidental pathogen release or containment breaches, which could trigger outbreaks or pandemics. As our world becomes increasingly interconnected, addressing the risks associated with these sensitive locations will be crucial to preventing future pandemics and safeguarding global public health.

Assessing Sensitive Locations as Potential Pandemic Sources
🔵 Bushmeat Markets required Heightened Surveillance, 🟢 High-Level Biohazard Laboratories (BHL3-4) and Wet Markets (not shown on this map for there are tens of thousands of them).
· Data for biolaboratories: https://www.globalbiolabs.org/map
· Data for bushmeat markets: Jagadesh S, Zhao C, Mulchandani R, Van Boeckel TP. Mapping Global Bushmeat Activities to Improve Zoonotic Spillover Surveillance by Using Geospatial Modeling. Emerg Infect Dis. 2023;29(4):742-750. https://doi.org/10.3201/eid2904.221022

 

☄️ From Above 🦕

Meteors & Meteorites

Between 2000 and 2013, a comprehensive network of sensors, designed to monitor Earth continuously for the infrasound signature of nuclear detonations, detected 26 explosions with energy levels ranging from 1 to 600 kilotons. Surprisingly, these explosions were not caused by nuclear activities, but by asteroid impacts on Earth, emphasising the often-underestimated threat posed by such events. The Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO) disclosed this data in 2014, highlighting the crucial role of their global network of sensors in detecting not only nuclear tests but also other significant events, such as asteroid impacts.

Impacted Earth
Between 2000 and 2013, a comprehensive network of sensors, designed to monitor Earth continuously for the infrasound signature of nuclear detonations, detected 26 explosions with energy levels ranging from 1 to 600 kilotons. Surprisingly, these explosions were not caused by nuclear activities, but by asteroid impacts on Earth, emphasising the often-underestimated threat posed by such events. The Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO) disclosed this data in 2014, highlighting the crucial role of their global network of sensors in detecting not only nuclear tests but also other significant events, such as asteroid impacts. These atmospheric impacts serve as a reminder of the potential dangers our planet faces, which can have considerable consequences and warrant increased awareness and preparedness.

Meteorites, Impacts and Fireballs
The 🟡🟠🔴 coloured hemispheres represent all known craters created by meteorite impacts, marking the remnants of these powerful collisions on Earth's surface. ⚪️ Tiny white dots indicate recorded meteorites discovered on the ground, tangible evidence of these extraterrestrial visitors. Meanwhile, 🔵 light blue circles signify fireballs that were observed in the atmosphere between 1988 and 2023, capturing the fleeting yet spectacular displays of these celestial events.
A significant event to mention is the Chelyabinsk meteor (large blue circle) that penetrated Earth's atmosphere in 2013, above the Southern Ural region in Russia.
Data: PASSC Earth Impact Dataset | Meteoritical Bulletin Database | Center for Near Earth Object Studies · Fireball and Bolide Data

Chicxulub Impact's Devastating Worldwide Tsunami
The Chicxulub asteroid impact 66 million years ago caused a massive global tsunami, which is seen modelled here. The vivid colours indicate waves over 10m high.
Source: Range, M. M., Arbic, B. K., Johnson, B. C., Moore, T. C., Titov, V., Adcroft, A. J., et al. (2022). The Chicxulub impact produced a powerful global tsunami. AGU Advances, 3, e2021AV000627. https://doi.org/10.1029/2021AV000627

 

🌋 And Below 🌊

Supervolcanoes

Supervolcanic eruptions, although rare, pose a significant existential risk to humanity due to their potential to cause catastrophic global consequences. The scale of these eruptions is measured using the Volcanic Explosivity Index (VEI), with a VEI of 8 representing a supereruption that ejects more than 1,000 cubic kilometers of volcanic material.

The most recent supereruption on record took place approximately 74,000 years ago at Indonesia's Toba Caldera, an event of such magnitude that it likely triggered a volcanic winter lasting for several years and substantially reduced the global human population. Ranking as one of the most colossal volcanic occurrences in the past 25 million years, the Toba eruption had a VEI of 8 and expelled an estimated 2,800 cubic kilometers of volcanic matter. This eruption not only led to a prolonged volcanic winter, but also contributed to a significant decline in human populations and created genetic bottlenecks in multiple species.

The Taupo Volcanic Zone in New Zealand experienced the Oruanui eruption around 26,500 years ago, which is considered one of the most powerful volcanic events in the last 100,000 years. This eruption had a VEI of 8 and released an estimated 530 cubic kilometers of volcanic material, significantly altering the landscape of the North Island and contributing to a short-term global cooling effect.

On average, super eruptions are estimated to occur once every 50,000 to 100,000 years, although the intervals between such events are highly variable. The potential impacts of a super eruption include widespread destruction of ecosystems, disruption of global agriculture due to volcanic ash and aerosols, and long-term climate change from the release of large amounts of greenhouse gases.

Large Magnitude Explosive Volcanic Eruptions
This map illustrates the most significant volcanic eruptions that took place during the Quaternary period (the last 2.58 million years).
🟡 Yellow markers indicate eruptions with a Volcanic Explosivity Index (VEI) of 7, such as the Tambora eruption in 1815, the Krakatoa eruption in 1883, and the Santa Maria eruption in 1902.
🔴 Red markers represent even more powerful eruptions with a VEI of 8, including the Toba eruption around 74,000 years ago, the Yellowstone eruption around 640,000 years ago, and the Taupo eruption approximately 26,500 years ago.
Data: Large Magnitude Explosive Volcanic Eruptions (LaMEVE) Dataset

The Yellowstone Caldera in Wyoming, United States, has experienced three major eruptions in the past 2.1 million years, with the most recent one occurring around 640,000 years ago. This eruption ejected approximately 1,000 cubic kilometers of volcanic material and created the Lava Creek Tuff, a rock formation covering large areas in the western United States.

Volcanic Calderas
These spikes illustrate volcanic calderas, i.e. are large, bowl-shaped depressions formed by the collapse of a volcanic edifice, typically following a massive eruption. The Yellowstone Caldera (55x72 km) in Wyoming, the Valles Caldera (22 km) in New Mexico, and the Long Valley Caldera (32x16 km) in California are clearly visible in this visualisation.
Data: Large Magnitude Explosive Volcanic Eruptions (LaMEVE) Dataset

Large igneous provinces (LIPs) are vast regions of the Earth's crust characterized by extensive volcanic activity that produces immense volumes of basaltic lava. These geological formations result from massive outpourings of molten rock, often associated with mantle plumes or other deep-seated sources. Two well-known LIPs are the Siberian Traps and the Deccan Traps. The Siberian Traps, located in present-day Russia, were formed around 252 million years ago during the end-Permian extinction event. The Deccan Traps, found in modern-day India, were created about 66 million years ago, coinciding with the end-Cretaceous extinction event. Both LIPs have been implicated in contributing to mass extinctions due to their vast emissions of volcanic gases, such as carbon dioxide and sulfur dioxide, which led to rapid climate change, ocean acidification, and widespread environmental disruptions.

Large Igneous Provinces (LIPs)
In this image, the Siberian Traps, located in present-day Siberia, and the Deccan Traps, situated in contemporary India, are especially noteworthy. Both these geological formations played significant roles in causing mass extinctions.
Source: Whittaker, J., Afonso, J., Masterton, S., Müller, R., Wessel, P., Williams, S., and Seton, M., 2015, Long-term interaction between mid-ocean ridges and mantle plumes: Nature Geoscience, v. 8, no. 6, p. 479-483, doi: 10.1038/ngeo2437.

In December 2021, the Hunga Tonga-Hunga Ha'apai submarine volcano in the Tongan archipelago began erupting, culminating in a massive climax on January 15, 2022. Part of the highly active Tonga-Kermadec Islands volcanic arc, this VEI-5 rated eruption, dubbed a "magma hammer," displaced 10 cubic kilometers of material and generated the largest atmospheric explosion recorded by modern instruments. The eruption triggered tsunamis throughout the Pacific, causing casualties and destruction in several countries. It was the largest volcanic eruption since Mount Pinatubo in 1991, and the most powerful since Krakatoa in 1883, with NASA comparing its power to hundreds of times that of the Hiroshima atomic bomb. On the other hand, supervolcanoes are classified as VEI-8, representing a significantly higher level of volcanic explosivity compared to the Hunga Tonga-Hunga Ha'apai eruption.

A Once-in-a-Century Event
The Hunga Tonga-Hunga Ha'apai volcanic eruption on January 15, 2022, had a force approximately 1,000 times greater than the Hiroshima bomb in 1945, making it comparable to the most potent nuclear weapon ever detonated by the US, the 1954 Castle Bravo bomb at Bikini Atoll.
Source: Himawari 8

Volcanic calderas (indicated by red spikes), large igneous provinces or LIPs (depicted as red areas), tectonic boundaries (represented by red lines) and earthquakes (dynamic white dots) all serve as reminders of the Earth's dynamic and occasionally violent nature, emphasising the significant risks posed by geohazards to our existence.

 

References:

Organisations Working on X-Risks: