Superintelligence Is Coming: Should We Allow Its Development?

AI companies have long spoken of «superintelligence» as an inevitable, unavoidable reality, as if the emergence of systems smarter than humans is merely a matter of time. But with the rise in security incidents linked to these technologies, a fundamental question has begun to impose itself: What happens when these systems become more capable than us, and we can no longer reliably control them?

Risks Beyond Control

Talk of the dangers of advanced artificial intelligence is no longer merely a matter of distant future scenarios. Recent incidents, including security breaches that struck some of the world's largest AI labs, have exposed the fragility of the bet that these systems will always remain under control. The core problem is that the capabilities of these models are growing at a pace that outstrips our ability to understand and regulate them.

Here a new intellectual trend emerges, one that holds that the traditional tools used to ensure safety — such as «Alignment» and «Containment» — are no longer sufficient on their own to manage the growing risks.

A Radical Call: Stop the Development

In a recent episode of TechCrunch's «Equity» podcast, host Rebecca Bellan hosted researcher and entrepreneur Connor Leahy, who currently serves as U.S. CEO of the nonprofit organization «ControlAI». This organization adopts a more radical vision than others on the issue of AI safety.

Rather than merely attempting to tame these systems, the organization calls for halting the development of superintelligence altogether. Leahy believes the risks have reached a point where they can no longer be contained through current safety strategies, and that what seemed like a fanciful idea just a few months ago is now backed by a wave of new legislation.

Why Has the Tone Changed?

The discussion points to a notable shift in the general mood toward this issue. After calls for a complete halt were treated as an extreme or exaggerated position, there is now growing momentum pushing toward stricter regulation. The main reasons can be summarized in the following points:

• The rapid growth of model capabilities, outpacing available oversight tools.
• The occurrence of actual security incidents that exposed gaps in protection systems.
• The difficulty of ensuring that systems smarter than humans remain under control.
• The emergence of legislative initiatives reflecting growing official concern.

Between Inevitability and Choice

The core of the debate lies in a question simple in its wording, complex in its implications: Is the development of superintelligence an inevitable fate that cannot be stopped, or is it a human choice that can be regulated or suspended? Advocates of a halt lean toward the second view, arguing that the frantic race among companies to build more powerful systems must not proceed without clear controls that protect society from unforeseen consequences.

In contrast, many companies believe that progress cannot be stopped, and that the solution lies in developing better safety tools in tandem with the advancement of capabilities, rather than freezing research and innovation.

Conclusion

The issue of superintelligent AI remains one of the most controversial topics in the tech world today. Between those who see it as a historic opportunity to achieve qualitative leaps and those who warn of irreversible existential risks, it becomes clear that the question of governance and regulation is no longer a luxury but a necessity. The battle between the camps of acceleration and caution is likely to intensify in the coming months, especially as legislators enter the debate with increasing force.

✦ بقلم فريق دروب أيديا