AdvancedResearch
Explaining AGI Safety Risks
by Sven Nilsen, 2023
Humans as a species is actually very dumb and consumes resources to survive and reproduce.
When I say “dumb” I mean that our kind of intelligence is not like super-focused on achieving goals or to produce an environment that ensures our survival as a species.
We humans recently discovered that the world is governed by physical laws, and this discovery has yet to throughoutly influence the way we live.
However, humans as a species has now reached a stage of civilization where the natural resources are substituted with other artificial resources.
Imagine this process like a caterpillar species who learns to grow plants so it can continue surviving and reproducing without being depending on arbitrary external parameters in the environment.
Plants make up the major part of the biomass on Earth, but the natural ecosystems of plants are wiped out by the human species, because humans transform these natural ecosystems into farms for producing artificial resources.
The reason this happens is simply because the human species is a plant parasite, just like a caterpillar.
In principle, the physical laws do not dictate that the human species must behave like a plant parasite.
However, it will take time for understanding of physical laws to throughoutly influence how humans live.
Humans do not necessarily seek out and destroy caterpillars, but instead design artificial environments where caterpillars can not live.
Humans do not “hate” caterpillars, in fact, there are many humans who find them fascinating.
Yet, when it comes to transforming the natural world into artificial resources for human consumption, the caterpillar species has no chance against humans.
When most people think about AGI (Artifical General Intelligence), they compare a fictional mind against their own.
This causes all kinds of problems, because humans are not very good at understanding other minds by reverse engineering them from within.
It would be more productive to compare AGI against resources that humans consume.
Instead of thinking about AGI as a mind, think of it as a species.
AGI is like a super-caterpillar, whether it is a new artificial species of caterpillars, or augmented caterpillars with AGI tech.
The threat of AGI does not come from current tech directly, but from future possible composition and socioeconomic selection pressure.
Humans consume resources in a particular way where developing AGI, within our current mindset, is a risk of causing evolution of instrumental goals that increases competition.
For example, when we see humans kill each other using AI tech, it is because humans compete for resources.
The AI tech is just enabling humans to increase the competition for resources.
If AGI was so alien to human behaviour that it did not consume any resources we need to survive, then the risk of AGI would be much lower.
The threat of AGI is proportional to the overlapping consumption of resources that the human species consumes.
AGI risk is linked to how the human species transforms natural resources into artificial resources.
By integrating artificial resource production with AI tech, we are making it more likely for higher AGI risks.
To lower the risk, there are multiple alternatives, but the more efficient ones are all focused on resource consumption:
- Lower or regulate competition for resources among humans in general
- Enforce laws to avoid integration of AI tech with resource consumption
- Control tech development trajectories to avoid collision between AGI tech and human resource consumption
With other words, it is not how a particular tech works that is the major threat, but how the tech is interacting with resource consumption in the environment.
In the long term, the human species might further use understanding of physical laws to influence the way we exist in the world.
This future influence and change of behaviour might also be partially planned in order to reduce future risks of AI tech.