Open problem 8
The Corporate Governance Problem
- frontier AI
- transformative AI
- AGI
Can corporate boards, safety teams, investors, employees, auditors, and internal policies reliably constrain frontier developers when commercial and strategic incentives point toward speed?
Why it’s foundational
Frontier AI governance is currently heavily dependent on the behaviour of a small number of firms. Voluntary safety frameworks, internal preparedness policies, red teams, and system cards matter only if internal governance can resist pressure.
Why it’s difficult
Corporate governance is built around control rights, fiduciary duties, capital markets, executive incentives, trade secrets, and labour mobility. Safety teams may lack veto power. Boards may lack technical understanding. Investors may discount catastrophic risk.
Hidden assumptions
Many proposals assume that “lab governance” is a real constraint merely because a lab has a policy. That is safety-washing unless the policy has authority, independence, resources, and consequences.
Competing positions
- Voluntary commitments
- Mandatory safety officers
- Board-level safety committees
- Public-interest directors
- Fiduciary-duty reform
- Licensing of frontier developers
- Public or non-profit development
- Nationalisation in emergencies
What could make progress
Empirical study of safety escalation inside labs; whistleblower-protection analysis; audit of board expertise and authority; investor-governance research; comparison with nuclear, aviation, pharmaceutical, and financial risk committees.
What it would change
It would decide whether policy should rely on internal safety frameworks or instead impose external licensing, liability, audit, capital, and governance requirements.
Sub-agenda
- What internal powers must a safety function have to be meaningful?
- Should safety officers have legal duties to regulators?
- How should fiduciary duties treat catastrophic-risk externalities?
- What board structures reduce race-to-release incentives?
- What whistleblower rights are needed for frontier AI?
Priority (editor scoring)
A small number of firms currently mediate much of the frontier risk.
- Importance
- 4/5
- Neglected
- 4/5
- Difficulty
- 4/5
- Actionable
- 5/5
- Robust
- 4/5
- Nat’l+int’l
- 4/5
Where the catalog bears on this
Instruments this puzzle names that are in the catalog:
Editorial content — a human-authored agenda question, rendered verbatim. No part of this analysis is AI-generated (see the charter).