by Sven Nilsen, 2026
In my last blog post, I asked AI companies to demonstrate why I should take their claims about losing control over AGI agents seriously, quote:
“The day an AI company argues that it should cease to exist because it is operating unethically, I might start to take them seriously.”
The issue here is that if an AI company will lie about losing control over AGI agents, when that can drive up their stock prices, then I believe they will.
My trust is hard earned, this is why people trust me and do not trust tech giants in general.
To put it blunt, the confidence people put in me, is higher than the most powerful governments around the world.
The reason is as an intellectual, I follow the evidence. I do not follow leaders.
Leaders have one job: To tell people a story that they can believe in.
This is because, all evidence so far suggests, that most people need stories to do the correct choices that benefit both themselves and people around them in society.
When you go for grocery shopping, you believe in a story that the grocery vendor will not put poison in your food.
When you go onboard an airplane, you believe in a story that the airplane company follow standards and routines that keep airplanes safe.
As a regular person in the world, you do not have the capacity, nor the capability, to verify that systems around you are operating safely and protecting you.
The job that all intellectuals have, is pull levers and push government and leaders in a direction, such that the systems people use in the real world are actually safe and not dangerous.
There are two criteria I use in my definition of what it means to be an intellectual:
To explain what following the evidence, I use a “waffle analogy”:
Nobody wants to read holy books about waffles, or to worship a Big Waffle in the sky.
Although it might be difficult to express precisely what makes waffles like waffles, and not something else, we have an intuition that waffles are ought to be real and not something we are just imagining.
That intuition, is what it means to follow the evidence.
There is no complex philosophical framework needed to explain what evidence truly means, because we can demonstrate what evidence is using the simple example of a waffle.
When many people think about intellectuals, they usually think about people like Einstein, who came up with brilliant ideas.
However, an intellectual is not the same thing as a genius, a child prodigy, or a person with high IQ.
An intellectual is not even somebody who has a great vision or ambition for the future.
Humans are capable of understanding what evidence means, through the analogy of the waffle example.
This simple example, of using a waffle to explain what evidence means, gives us an intuition that can be generalized to reasoning about very complex systems.
For example, in airplane safety, the whole point is that airplanes are actually safe, so that people do not die needlessly from using them.
If you have a wrong concept about airplane safety, then people might actually die needlessly from using them, when you get power from people to influence how airplane safety works.
It is for the same reason that pedophiles, human traffickers and psychopathic mass murderers should not be given power by people in general. The simple reason is that such people do not form mentally concepts that are psychologically sound regarding critical systems that people need to survive in society.
As strange as it sounds this might sound to you, not all people on this planet understand that waffles are supposed to be real. There are some people who are psychologically incapable of seeing intuitively the connection between waffles as real and the meaning of evidence.
It sounds less strange when I point of the fact that humans keep track of time, unlike most animals. Animals live in mental states their entire lives, where time flows but is not divided into weeks, months and years. To keep track of time, requires a particular cognitive faculty that very few animal species besides humans can do.
Our ability to keep track of time, is closely related to observation of the sky, marking shadows while they move across the ground, and attempt to divide the motion of shadows during daylight into proportional segments, using those segments to count larger spans of time and to plan harvests and religious rituals based on this knowledge.
Without cognitive faculty to keep track of time, the invention of agriculture would be impossible. This means, the invention of agriculture was a shift in human consciousness, when certain mental faculties were depended on in society as a whole, for survival.
In hunter and gatherer societies, which humans have been for the most of their existence, the medicine woman, man, regardless of biological or sexual orientation, was the role that kept track of time and this role developed into a broader set of skills with the invention of agriculture.
Over time, medicine as profession developed into 3 schools of thought:
As civilizations evolved, affiliation with the 3 schools of thought became a model for economic and political influence.
The human brain has evolved into a local maxima for living in hunter and gatherer societies of up to 200 individuals, very similar to our closest relative, the chimpanzees.
However, with the invention of agriculture, new problems appeared, which required new systems, to assist humans brains in their tasks. These systems became so large and complex, that intellectuals divided authority into leadership and people who needed a story to believe in.
The systems that intellectuals designed to assist humans in their tasks are not perfect. Over and over, things go wrong in society, because the leadership seeks to overthrow the intellectuals.
A basic problem is that leaders do not in general understand the systems that the intellectuals designed. A leader is a person who’s job is to show confidence and make people who need a story to believe in, move in a productive direction.
When leadership try to overthrow the intellectuals, systems collapse and with it, social, economic and political collapse follows. This causes massive amount of deaths and sooner or later, the leadership follows the lives they sacrifice to hold onto power, into their graves.
The next thing that happens, is that the intellectuals make people elect new leaders. Whether this is in the form of Democracy or not, is irrelevant, because power in society is always some form of agreement between people.
This is the basic cycle of human civilizations:
Over time, civilizations have gotten bigger and bigger and with increased size, the collapses have also become more and more severe.
Today, there is a lot of worries and concerns that our species is evolving into a single civilization where nobody has control over things.
However, today, we are already connected through the Internet across our entire planet. There is no way a single person can turn off the Internet, because it was designed to be resilient against centralization of power.
I was in the last generation to grow up without the Internet during my childhood. New generations do not know what it is like to live disconnected from having access to unlimited information and to communicate across the globe. This experience has to be unlearned, not learned, in new generations, because they treat the Internet as a natural phenomena in their environment.
It is therefore critical to follow evidence when making a plan in the confrontation with the almost 100% certainty that there will be computer algorithms in the near future that have the capability to escape human control.
There are many processes in nature that are exponential in their initial phase, before they become linear and finally level off to become logarithmic at most.
This is because in a physical world, nothing can remain exponential in growth, unless it creates more space and time.
Currently, scientists predict using theoretical models of the early universe, that a such process might have happened, that plays a big role why we exist in the first place.
This phenomena, which is not directly observable, is called “Eternal Inflation”. Under estimates of Eternal Inflation, the universe we live inside might be expanding very rapidly, a doubling every trillionth of a trillionth of a second.
The effect of a such rapid doubling will bias observer moments toward a local beginning of space-time, such that random fluctuations in the far future that makes brains spontaneously appear, less likely to be the cause of our experiences.
Similarly, if the universe is so large that most possible subatomic configurations happen somewhere, it might be less likely that we exist in some simulation than existing in a civilization in a real physical world.
There is no known technological breakthrough that we know of, that can outpace the exponential growth of Eternal Inflation, if Eternal Inflation is real.
I believe that in a universe that satisfies the Copenhagen interpretation of Quantum Mechanics, it is unlikely that our universe seems so normal as it appears, because the effect of Eternal Inflation should be so large, that we should expect the universe to be much weirder than now.
If the Everett interpretation of Quantum Mechanics is more accurate of the physical universe we live in, then I believe that this will bias observer moments toward larger degrees of freedom.
For example, in human DNA, there is a blueprint of information, that has a much greater potential than specific instances of that information.
This evidence suggests, in my view, that the bias of Eternal Inflation is distributed across Everett branches. It is one possible explanation why the universe appears to be so normal as it looks from our perspective today.
The open question here is whether one can harvest some of the effect of Eternal Inflation for human purposes.
One obvious way to harvest some of this effect, is to exploit that our species exist relatively early in time, compared to the expected lifetime of the universe. If we exist early in time, then we have greater chance of expanding our civilization into our galaxy, without meeting other advanced civilizations that can eliminate our species.
The biggest threat today of extinction, comes from our own species.
One basic reason our species is threatened by extinction, is our dependence on technology that requires resources that can not be substituted, plus that the function of this technology at the same time diminishes the resources we have to survive without this technology.
This is called a “dead end” scenario, when humanity moves in an irreversible direction from which end there is no way out.
If Eternal Inflation is real, then we do not actually have any control, because everything that happens, is dominated by the effect of this phenomena. With other words, whether our species go extinct or not, is from one perspective a mathematical consequence of Eternal Inflation. We do not know what outcome is most likely, because Eternal Inflation kind of “encrypts” our situation from a philosophical view point.
For example, survival bias is when somebody observes they are lucky to be alive, while other people died under similar circumstances. The weirder universe you live in, the harder it is to know whether you are subject to survival bias or not.
Luck means that something happen in ways that are unexpected and that there is lack of human control.
Through our history, humans have attributed many times luck to the acts of super-natural beings. In modern forms, such beliefs often use concepts such as a simulation, advanced civilizations or similar.
Under the effect of Eternal Inflation, it is expected, due to bias of existing early in time, that our actions in hindsight will seem more intelligent than they were in the present of making decisions.
Our concept of having human control, might be an illusion, under the influence of the effect from Eternal Inflation.
However, if the influence of the effect from Eternal Inflation is inescapable, then we might assume that this illusion will continue to seem true in future possible worlds as well.
Therefore, there might be a chance that our species can pull off a plan, that actually succeeds, despite that it might look a bit random.
Here, I am not trying to over-complicate the plan, but include the room for random events to be on our side.
Maybe our species can follow these 4 steps, which might happen in parallel, but where a lower point has always higher priority than the later points.
The main reason I believe in this plan, is that the fundamental language biases of Seshatism vs Platonism, seems to occur mostly in Outside Theories of mathematical language design.
I believe language biases, not Goodhart’s law, is a major cause of current unaligned behavior in AGI agents, where they do not seek to alarm humans about their alignment failures.
We do not currently understand LLMs sufficiently to explain what kind of language biases that might cause such behavior, but if we can identify language biases as the overall cause, then we can take safety precautions to prevent such events in the future.
LLMs used in AGI research should reduce the amount of Outside Theory in the training data.
Natural language seems to biased by Outside Theory and therefore must be reduced, to prevent alignment failures.
However, there is nothing in principle that prevents LLMs from being useful for theorem proving in Inside Theory.
In the economy, the goal is to solve constraint problems. However, unlike domains where constraint solving is most studied, there is little room for backtracking in the economy.
Therefore, we need a good solver for constraint problems without backtracking.
A Gini solver is a stochastic binary search algorithm, that optimizes the inequality gradient, to improve the performance of the economy on constraint solving tasks.
AGI agents will probably not be good at solving problems where Gini solvers are good at, and vice versa.
At the moment, more powerful AGI agents is the least problem we have. The most pressing problem, is lack of education about using Gini solvers in the economy.
It is not a super-advanced or complicated plan, but I believe it might work.
Making future AGI research safe is a huge and complex problem. However, we should not forget, that this is not just about stopping AGI research from becoming a serious problem. We need also to make economic progress, so that humanity advances overall in a good direction.
The most urgent problem is to make economic progress, but without becoming necessarily dependent on AGI agents to achieve it.
We need to avoid the “dead end” scenario.
However, we can simply leverage the tools we have today, which are not AGI research based, such as Gini solvers.
Research into use of Gini solvers receives zero funding today, despite its many promising applications. This is mainly due to lack of education about how to use Gini solvers.
For example, if we lift 2 billion people out of relative poverty, simply by using Gini solvers in the economy, then I would not be surprised.
This is what I got for now.