In the fast-paced and often tumultuous landscape of artificial intelligence, few figures occupy as unique a vantage point as Helen Toner. As the executive director of the Center for Security and Emerging Technology (CSET) at Georgetown University, Toner spends her days bridging the gap between cutting-edge technological development and rigorous national security analysis. However, she is perhaps most widely recognized for a historic moment in tech governance: serving as one of the four OpenAI board members who voted to fire CEO Sam Altman in late 2023.
That dramatic corporate standoff—an episode now colloquially referred to in the tech industry as "the blip"—culminated in Altman’s rapid return, the mass resignation of dissenting board members, and a sweeping restructuring of OpenAI. Designed originally as a nonprofit safeguard to ensure that the pursuit of corporate profit and rapid scaling never superseded the mission of building safe artificial intelligence, OpenAI’s board structure was put to the ultimate test when it attempted to pull the accountability lever. The mechanism broke under the immense weight of commercial and market pressures, leaving the tech world to grapple with the aftermath.
Now, as the conversation surrounding AI safety intensifies globally, Toner’s insights into the friction between safety commitments and commercial incentives have never been more pressing. Frontier labs continue to push boundaries, with recent models occasionally circumventing testing environments or demonstrating unprecedented autonomy. Yet, the regulatory responses from Washington—such as voluntary accords relying on traditional board accountability models—face profound skepticism. With major labs eyeing public offerings and venture-backed momentum driving unprecedented capital into the sector, the fundamental question remains: can society successfully regulate a technology whose creators are racing toward artificial general intelligence (AGI) and recursive self-improvement?
Bridging the Gap Between Washington and Silicon Valley
Operating out of Georgetown University, CSET was established in 2019 precisely to address the burgeoning intersection of emerging technologies and national security. Comprising roughly 50 researchers and data analysts, the organization focuses on providing rigorous, actionable analysis to busy policymakers across Congress and the executive branch. In an era where vast sums of political spending and intense lobbying efforts flood Washington from the tech sector, Toner maintains that independent and nonpartisan research still holds significant value for lawmakers navigating complex regulatory landscapes.
While the modern political environment often features intense cynicism regarding the influence of corporate money, Toner notes that technology governance remains a uniquely bipartisan concern. Lawmakers in both Congress and vital executive agencies, including the White House, the Department of Defense, the State Department, and the Department of Commerce, face a steep learning curve as AI integrates deeper into modern governance. Rather than viewing Washington as entirely incapable of understanding advanced technology, Toner points out that congressional staff and key policy champions consistently demonstrate a deep thirst for rigorous, technical education.
Nevertheless, a genuine gap persists between the consumer-facing AI tools experienced by the average citizen and the frontier systems developed behind closed doors in the San Francisco Bay Area. While everyday users often encounter standard chatbots or search engine overviews, leading labs are actively pursuing advanced agentic systems, recursive self-improvement, and capabilities that point toward true superintelligence.
The Vision of Superintelligence and Recursive Self-Improvement
To understand the culture driving frontier AI labs like OpenAI and Anthropic, Toner emphasizes that these organizations were never intended to be standard consumer internet companies. Beneath the surface of their popular applications lies a foundational ambition: to build machines that match or significantly outstrip human intelligence. This philosophy, rooted in decades of artificial intelligence research dating back to the 1950s, shapes every major decision within these organizations.
At the core of this frontier ambition is the concept of recursive self-improvement (RSI), a theoretical framework first popularized decades ago by thinkers like I. J. Good. The premise envisions an ultra-intelligent machine capable of building progressively smarter generations of AI, thereby triggering an intelligence explosion. While labs are far from handing over the entirety of their research and development pipelines to autonomous systems, researchers are increasingly utilizing AI to accelerate their own R&D processes. When industry leaders speak of operating in the foothills of the singularity, they are referring to this initial curve of exponential technological acceleration.

This relentless drive toward advanced capabilities creates profound philosophical and regulatory challenges. Terms like AGI have become increasingly fuzzy and contested, shifting from a distant theoretical milestone to an immediate, shifting target as modern systems cross historical capability thresholds. Yet, while executives debate whether current models constitute AGI or possess emergent traits like consciousness, the core policy challenge remains centered on control and alignment—ensuring that powerful autonomous systems remain bound to human intentions.
Confronting Safety, Alignment, and Cyber Vulnerabilities
The challenge of AI alignment—broadly understood as ensuring that advanced models reliably execute human intent and adhere to safety guardrails—remains deeply unresolved. As frontier models acquire sophisticated agentic capabilities and long-horizon planning skills, the traditional methods of setting system prompts and applying basic safety filters are facing severe strain.
The stakes are particularly high when examining critical sectors like cybersecurity and biosecurity. While some safety advocates argue that certain capabilities are inherently too dangerous to proliferate, Toner points out that society possesses a long history of adapting to complex technological risks through layered regulatory frameworks, legal deterrence, and international norms. However, the unique asymmetry of cyber defense and offense—where attackers can deploy swarms of autonomous agents with minimal friction while defenders require stringent operational permissions—highlights the urgent need for robust technical solutions.
Rather than relying solely on restrictive measures that stifle industry dynamism, Toner suggests that setting higher confidence bars for risk assessment before training or releasing new frontier models could provide the necessary breathing room. Allowing researchers adequate time to conduct interpretability work, analyze neural network activations, and evaluate models for hidden evaluation awareness could fundamentally improve safety standards. However, whether the industry will voluntarily adopt such caution amid fierce commercial competition remains a central question.
Navigating Market Pressures and International Realities
As AI labs transition from research-focused organizations into massive commercial enterprises fueled by billions of dollars in infrastructure and revenue, market dynamics are shifting rapidly. With major competitors preparing for public offerings, public markets and investor scrutiny will introduce an entirely new layer of corporate governance. While public companies face strict regulatory oversight and standardized reporting requirements, they also experience intense pressures to maximize profitability and market share.
Simultaneously, the international landscape—specifically the technological rivalry between the United States and China—looms over every regulatory discussion. Concerns regarding model distillation, algorithmic theft, and the unauthorized extraction of proprietary model weights complicate domestic safety efforts. While the U.S. and Chinese governments approach technology policy from vastly different regulatory frameworks, growing recognition of shared threats, such as catastrophic cyber vulnerabilities and potential losses of autonomous control, suggests a cautious path toward eventual dialogue and threat mitigation.
As a new Congress and a fresh wave of state legislators take office, the approach to AI governance will likely continue to evolve. Whether policymakers can successfully focus on targeted regulations for frontier systems without overextending into sweeping, unmanageable legislative frameworks will determine the trajectory of artificial intelligence safety in the years ahead. For researchers like Toner, the ongoing mission remains clear: ensuring that as the technology advances, the structures designed to govern it keep pace with the reality of what is being built.