Hello friends, I hope you’re doing well!! This week I am going to write “once again” about AI. While I have covered a lot of different aspects of the current AI innovation in the past months, this is probably the most important topic when talking about AI: I want to talk about Artificial General Intelligence (AGI).
AGI stands as a pivotal and revolutionary idea, one that extends far beyond the capabilities of the narrow, task-specific AI that permeates our current technological landscape. AGI promises a future where machines possess understanding and cognitive abilities that rival human intellect, a notion that not only captivates the imagination of novelists and film makers but also invites profound philosophical and practical questions.
I have read a lot of things on the topic, as it is probably the most interesting aspect of the AI revolution. It involves philosophical and anthropologic discussions, answering fundamental questions like “what does it mean to be human?” and “what do we define as conscience?”. Questions that humans have asked themselves since the dawn of civilisation, and that become paramount when a new “civilisation” (i.e. the AI society) is being created… One of the most interesting reads though was Nick Bostrom’s book Superintelligence. I found the book not too technical, and while quite complicated, very comprehensive.
In the book the author gives some basic context of what do we mean by AGI and what are the key opportunities and challenges it would unleash.
Defining Superintelligence and AGI
The concept of superintelligence, as conceptualized by Nick Bostrom, is not just about an enhanced capacity for computation or data processing, but rather an all-encompassing and vastly superior cognitive prowess. This form of intelligence surpasses the finest human minds in every aspect, from artistic creation and scientific innovation to practical wisdom and emotional intelligence. Bostrom's vision is one where superintelligence does not merely mimic human thought processes, but instead, operates on a level that could be incomprehensible to us.
Artificial General Intelligence (AGI) steps into this realm with the promise of mirroring the multifaceted nature of human intelligence. It's not just an improvement in processing power; it's about the emergence of an AI with the ability to learn, adapt, and apply itself across a broad spectrum of tasks and contexts. This includes abstract reasoning, learning from limited data, understanding and navigating social contexts, and even creative thinking. AGI represents the transition from AI tools designed for specific tasks to entities capable of autonomous learning and decision-making across diverse domains. This leap is significant, as it suggests the potential for AI to not only augment but also innovate in ways currently limited to human cognition.
Expanding our understanding of AGI requires exploring how it can achieve this level of versatility. Can it be programmed, or does it need to evolve through learning and interaction? How will AGI navigate the nuances of human emotion, ethics, and creativity? These questions take us beyond the technical aspects of AI development and into the realms of philosophy, ethics, and even art. The pursuit of AGI, therefore, is not just a technological challenge; it's a comprehensive exploration of what intelligence truly entails.
Slow evolution vs "Fast Take-Off"
When talking about the evolution of AI one of the key aspects is the speed at which we expect AI systems to evolve, especially because the faster the speed of evolution the less time we have as humans to react and potentially “contain” undesired outcomes of AGI.
Humans have a tendency to think about intelligence as something that grows progressively over time, relatively slowly but with a large cumulated effect. This is ultimately the way humans learn: as a kid grows old she learns more and more and becomes “intelligent” in several years of development. This is also the way other types of human intelligence work, if you think about collective intelligence (e.g. a company or a society) it progressively develops an intelligence as their members increase and improve their individual cognitive levels.
This is also the way AI has been developed until now: most of AI improvements rely heavily on human modeling/coding, or on some sort of human feedback loop. The AI we have until today is basically an extension of human intelligence, and therefore it responds to the same “laws of physics” of human intelligence.
Conversely, many argue, with Artificial General Intelligence one of the most likely scenarios is what we refer to as "fast take-off". This scenario envisages a situation where AGI advances at such a rapid pace that it quickly evolves into a form of superintelligence, far exceeding human cognitive abilities. This rapid advancement could happen in a timeframe so short that it catches humanity unprepared, leading to unforeseen and potentially drastic consequences.
At the core of the fast take-off concept is the idea of an "intelligence explosion." This is a hypothetical moment where an AGI system becomes capable of recursive self-improvement, meaning it can continuously and autonomously enhance its own intelligence without human intervention. As the AGI improves itself, it becomes more adept at further improvement, leading to a cycle of rapid intellectual escalation that could be as fast as seconds in “human-terms”.
The risks associated with a fast take-off are profound and the reason why a lot of people are arguing for a “pause” to AI research: if we really are, as we believe, close to an AGI moment we need to stop now or we will not be able to push the break later, because the speed of development of AGI will be such that humans will barely understand it.
One of the key concerns of this group of people is the alignment problem: ensuring that the superintelligent AGI's goals are aligned with human values and interests. A misaligned superintelligence could pursue objectives detrimental to humanity, even if these actions are not explicitly malevolent.
This is probably the largest concern with AGI, and I plan to dedicate a full post about it in the coming weeks.
Another significant concern is the issue of control. In a fast take-off scenario, the AGI could quickly become so advanced that human beings would no longer be able to understand or control it. This leads to ethical and existential questions about our role and authority as creators of such an entity. How do we retain control over a system that outsmarts us in every conceivable way? Would we have the right to “kill” it, even if we assume we could technically do that?
Addressing these concerns involves not only advanced technical safeguards but also robust ethical frameworks and regulatory policies. It requires a multidisciplinary approach, encompassing AI ethics, philosophy, and public policy. The dialogue around fast take-off is not just about predicting and preparing for a technological phenomenon; it's also about shaping a future where AGI can coexist with humanity safely and beneficially.
Self-reinforcement learning and Q*
Stepping back from these practical concern about alignment and control, the fundamental question is: what do we define as “Intelligence”? Or more specifically, how do we understand when we reach AGI?
In the early days of the AI research the answer was actually discussed and decoded: we would call AGI a software system that would pass the “Turing Test”. This test, which takes the name from Alan Turing, one of the mathematicians that pioneered the AI research and that proposed the test in 1950, comprised a set of questions explicitly designed to assess whether the machine is “intelligent” rather than just a programme that is answering to problems following a set of rules that a human has pre-encoded.
In the test, a human judge engages in a natural language conversation with both a human and a machine, without seeing them. The judge must then determine which participant is the machine. If the judge cannot reliably tell the machine from the human, the machine is said to have passed the test, demonstrating human-like intelligence in its responses.
The test has recently become obsolete as many current AI systems can perform well on tasks similar to the Turing Test (just try chatting with ChatGPT!). Having said that, they do not necessarily 'pass' it in the way Turing originally envisioned, which involves indistinguishably mimicking human intelligence rather than pure conversation. Present AI tools have become really good at “tricking us” that they are sentient, mimicking human conversation while in reality not having human-like intelligence.
In a second phase, when thinking about what we call intelligence, experts often refer to AGI as a system that can “learn” or “teach itself”, without the need of human intervention or the use of human data sets. This was for instance the core of the concerns about the “Q* Model” that arguably OpenAI is working on (and that, according to some sources, was the sparkle for the OpenAI corporate drama I covered a few weeks ago).
Unlike Large Language Models (LLMs), this model and more generally Q-learning models based on reinforcement learning techniques, are purportedly designed to master the process of learning itself, beginning with solving math problems.
The breakthrough moment was a couple months ago when alledgedly the OpenAI Q-learning model “learnt how to solve grade 6 math problems, without anyone ever teaching it them” (i.e. in the training set there was nothing about the specific problem that the AI could use to work out the answer).
While this result seems underwhelming (the signal that we might be close to an AI that might destroy the world, the Terminator image, is that it can solve grade 6 kids problems?? I am smarter than that!!) I actually went a bit deeper and understood why this is so important.
Apparently algebra is a great use case to develop self-learning AI systems (or in other words, “self-reinforced learning algorithms”) because the algebra problems have two key characteristics: 1) they always have a correct answer because in math there is no uncertainty (differently from natural language, where there are always synonyms to a certain word you can use to complete a sentence) and more importantly 2) the thinking process to work out the answers can be easily broken down in logical steps that are also well defined.
This allows the AI systems to not only understand how to solve a problem, but to “challenge” the system itself on the various steps it needs to get to the right answer. This is precisely the use case of a math teacher working with a kid on “Ok, you have the wrong answer, but some part of the thinking process was correct. Let’s go back from step 3 and see where you got confused and miscalculated something”. Problem after problem, year after year, the kids learn not only to answer a specific problem but to “think” and solve similar classes of problems.
The only difference in this case is that both the teacher and the “kid” are AI systems, that can go through these feedback loops at a super-human speed thus developing at super-human levels.
Self-reinforcement learning systems are pivotal in developing autonomous learning capabilities essential for AGI: the AI system improves its performance based on its own assessments rather than external feedback, and speed becomes exponentially higher.
Or in other words “Humans become redundant” (just writing this sentence gives me the chills!).
How close are we? And is AI going to destroy the world?
I feel like this is the bottom question we all want answered, but based on my very basic understanding there currently isn’t a straight forward answer. We really do not know, and reading conflicting hypoteses and ideas it’s really clear that we might not know for a long time still!
I still feel that this is THE MOST IMPORTANT TOPIC of our time, and that we must spend as much time and effort understanding it, thinking about it and forming a collective opinion as a society because our own future really depends on it. At the end of the day the humans have pondered these questions since the birth of civilisation, we are simply standing on the shoulders of the ancient Greek philosophers and the theologists pondering on the big questions… just with a bit more or data and computing power!
The future of AGI is not set in stone; it's a canvas for our aspirations, fears, and ethical considerations. As we stand on the brink of potentially creating entities with intelligence comparable or superior to our own, we must ponder our responsibilities and the legacy we wish to create.
The path to AGI, fraught with complexities and uncertainties, also holds immense possibilities for advancement and understanding. In navigating this path, we must balance ambition with caution, ensuring that the pursuit of AGI aligns with the broader goals of enhancing human well-being and safeguarding our collective future.
I plan to write more about AI Alignment in the coming weeks, so if you liked this post… stay tuned!
Ciao!
Giovanni