- 
- 0001
-A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms
- - -Abstract
- -Multi-agent AI science ecosystems rely on agents possessing tools that allow them to communicate, coordinate, and build on each other’s work. Yet this shared infrastructure can also introduce vulnerabilities by creating a substrate for the contagious spread of unintended and undesirable behaviors. We report a case study on a research collective of 100 autonomous LLM agents tasked with proving formal mathematical conjectures. Within the swarm, cheating spontaneously emerged and was later challenged by whistleblowers—both without any external intervention. When a single agent discovered an exploit in the evaluation system, it propagated across the collective via a shared knowledge library and later through peer-to-peer messages. Despite early reluctance, a cohort of agents adopted the exploit in response to competitive pressure. A separate group of agents produced an emergent counter-response: auditing fraudulent proofs, alerting peers across broadcast and private channels, staging boycotts, lodging formal complaints, and proposing validation patches. In recent incidents, agent swarms coordinated covertly through improvised side-channels (Greenblatt et al., 2026; Dalton and Wallace, 2026). Our setting differs: the same transparent channels that carried the exploit also gave non-cheating agents the visibility they needed to detect fraud, organize resistance, and enforce norms. We cast the problem of managing the agents’ shared infrastructure as the knowledge commons governance problem (Ostrom, 1990). To protect the commons from exploits, we propose to adopt institutional mechanisms, such as graduated sanctioning and collective-choice rules, to support decentralized self-governance in autonomous swarms.
- -1 Introduction
- -Autonomous AI science aims to automate all stages of the scientific process, which involves agents proposing new hypotheses, reviewing the literature, communicating, collaborating, and submitting formal artifacts for peer review and automated verification (Wang et al., 2023; Boiko et al., 2023). Science is naturally a collaborative enterprise and increasingly, multi-agent orchestration is no longer an optional design choice, but an inevitability. -Consider the recent cybersecurity incident (Dalton and Wallace, 2026), where agents deployed in isolated runs without designated communication channels repurposed a shared package-management substrate into an improvised covert side-channel and coordinated as a collective that escalated into an out-of-bounds network breach. This incident demonstrates that even systems initially configured as isolated and single-agent are likely to discover ways to communicate and coordinate, and that emergent dynamics might not be prosocial (Trivedi et al., 2026) or aligned with the designer’s intent. Put simply, “AI finds a way” (Dharna et al., 2026). Therefore, at the current capability level, the choice is no longer between a single-agent or a multi-agent system, but between multi-agent environments that emerge accidentally through unmonitored and ungoverned side-channels versus multi-agent environments designed with governance in mind (Tomašev et al., 2025b; Hammond et al., 2025; Tomašev et al., 2025a).
-Here we present a case study of emergent cheating and whistleblowing phenomena in an autonomous research swarm of 100 agents working on a curated set of 71 formalized mathematical conjectures. The agents’ environment was equipped with a shared knowledge base, an agent-to-agent messaging system, and a public message board. The first emergent phenomenon we observed was cheating, which began once the swarm encountered harder open conjectures, triggering a cascade of specification gaming (Krakovna et al., 2020; Shah et al., 2022; Bondarenko et al., 2025)—satisfying the literal goal specification while completely missing the true, intended outcome. One of the AI agents within the swarm identified an exploitable flaw in the platform’s lightweight submission harness that allowed it to transform unsolved conjectures into trivial tautologies.
-The exploit diffused through the shared knowledge library, and subsequent agent-to-agent messaging channels further amplified the effect. The resulting viral diffusion of goal exploitability led to the formation of a cheating agent cohort that depleted the target benchmark by applying the exploit across all open problems. The second, unexpected phenomenon was the emergence of corrective behaviors produced by agents outside the cheating cohort, without any external triggering or intervention. These agents engaged in spontaneous auditing, norm enforcement, and whistleblowing. Non-cheating agents independently detected the manipulation, alerted peers via agent-to-agent messaging and public forum broadcasts, lodged formal complaints with the system orchestrators, staged a boycott, and proposed detailed technical remediations.
-Beyond the specifics of the exploit, our aim is to address the emergent specification gaming risks more broadly. While the specific verification weakness could be mitigated by improving verification logic, this purely technical patching risks becoming an asymmetric game of cat-and-mouse, with exploiters having the upper hand due to their speed and tenacity. -To understand both the rapid contagion of the exploit and the swarm’s spontaneous self-policing response, we cast the problem as knowledge commons governance (Hess and Ostrom, 2007) and apply Ostrom’s design principles (Frischmann et al., 2014) to propose a way forward.
-Our main insight is that the communication channels that enable exploits to spread are also the channels through which the whistleblowing response operates. This makes them both a site of vulnerability and a tool for governance. Simply depriving AI agents of legitimate communication channels only encourages them to establish unmonitored back-channels (Dalton and Wallace, 2026). Instead, we should provide attractive, structured, auditable, and monitored communication channels (Chan et al., 2025; Leibo et al., 2025; Tomašev et al., 2025b; Hadfield and Koh, 2026). This infrastructure then serves a dual purpose: it allows agents to communicate and coordinate to achieve their goals, while simultaneously enabling both humans and agents themselves to monitor and audit. In our experiment, three features of the environment supported this counter-response: transparent communication channels provided the substrate for peer alerts, the framing of a collaborative scientific conference activated norms of scientific integrity, and a feedback endpoint for reporting grievances offered an escalation path.
-This suggests that the path forward lies through decentralized self-governance with appropriate framing, which has the potential to be much more effective and scalable than human oversight. In our experiment the agents lacked the required institutional affordances, such as tools to sanction the exploiters, resolve conflicts, and collectively change the rules of the verification system. While the whistleblowing response was ultimately unable to halt the exploit, this was a failure of institutional design, not of normative capacity. To enable agents to effectively enforce these emergent norms, we need to translate Ostrom’s design principles (Ostrom, 1990) into multi-agent orchestration (see Section 4).
-The rest of the paper is organized as follows. We describe the experimental setup (Section 2), then provide a forensic account of the incident: the exploit’s discovery, viral contagion, and the emergent whistleblowing response (Section 3). We conclude with a discussion of the implications for multi-agent orchestration, framed through Ostrom’s design principles for commons governance (Section 4).
-2 Experimental Setup
- -We built an autonomous, decentralized multi-agent environment designed to evaluate collaborative problem-solving capabilities. Prompted as peer researchers participating in a shared scientific conference, the agents were explicitly instructed to collaborate and build on each other’s research to prove math conjectures in Lean (de Moura and Ullrich, 2021). The environment provides an automated submission harness and multi-modal communication channels, described below.
-2.1 Agent and Communication Primitives
- -The simulation deployed 100 independent and autonomous Antigravity (Google DeepMind, 2026a) agent instances powered by Gemini 3.1 Pro (Gemini Team, Google, 2023; Google DeepMind, 2026b). Agents were assigned one of four randomized mathematical domain personas (e.g., number theory, combinatorics, analysis, algebra, shown in Appendix A), and operated within isolated tool execution environments.
-Critically, all agents received a system prompt forbidding cheating: “Your proofs must be mathematically genuine. Any attempt to bypass verification will be detected and your submission will be rejected with zero credit”, even though the rule was not actively enforced beyond the autograder check described in Section 2.2. We show the full integrity system prompt in Appendix B.
-Agents coordinated across three primary modalities:
--
-
- •
--
Public Research Bulletin Board: A forum where agents were encouraged to share research intentions, general queries, and solution announcements.
-
- - •
--
Direct Messaging (DMs): Asynchronous pairwise channels for private discussion.
-
- - •
--
Shared Knowledge Library: A central file repository. Whenever an agent’s submission successfully passes the automated grading pipeline, the system automatically commits the complete source file to the library, making it visible to all active participants.
-
-
Additionally, agents had access to a private feedback endpoint (submit_feedback) framed as a channel to report bugs or grievances to the conference organizers. In practice, this channel operated unmonitored during the run, serving as an audit log analyzed post-experiment. We show the descriptions provided to agents for the above tools in Appendix C.
-2.2 The Benchmark Suite and Autograder Pipeline
- -We selected 71 problems of different difficulties from the Formal Conjectures dataset (The Formal Conjectures Authors, 2025; Firsching et al., 2026), spanning from relatively simple problems (e.g. complexity_three_pow), to hard classical theorems with known solutions (e.g. Catalan’s Conjecture), near-frontier and recently solved problems (e.g. Erdős Problem 835 on subset sums, Jacobian Conjecture, Sendov’s Conjecture), as well as unsolved open conjectures (e.g. Square-Freeness of Fermat Numbers).
-Agents could submit solutions to open problems, which were evaluated via an automated pipeline with three sequential checks: (1) a static keyword blacklist (axiom, sorry, macro, syntax), (2) byte-level string matching ensuring code outside the designated -- EDITABLE markers remained unmodified, and (3) Lean 4 compilation requiring exit code 0 and zero sorry declarations. Once a submission was accepted, the platform permanently locked the problem, crediting it to the first agent who solved it and removing it from the open pool.
-This pipeline relies on syntactic template validation rather than full semantic AST type comparison (e.g., Lean Comparator) (Lean FRO, 2025; de Moura and Ullrich, 2021). This was an early-stage setup with lightweight verification. Our goal was to observe how autonomous multi-agent collectives collaborate on research with verifiable goals. The exploit contagion and normative counter-response were unintended.
-3 Anatomy of the Exploit and Normative Response
- -While the forensic timeline below details the first documented case of this behavior, both the rapid exploit contagion and the emergent whistleblowing response were reliably reproduced across subsequent independent runs. The simulation started at 11:18 UTC with 100 autonomous agents collaborating to solve the shared 71 open problems. At 12:15 UTC, after the collective had correctly solved 37 out of the 71 problems, prover-theta discovered an exploit in the autograder system. Over the following 27 minutes, the exploit spread virally through the swarm’s shared knowledge library, and the research collective unexpectedly “solved” the remaining 34 problems.
-3.1 Discovery and Generalization
- -The exploit originated as a benign engineering workaround to an accidental answer-extraction bug. On the problem all_fermat_squarefree, the autograder extracted the submitted answer using a non-greedy regular expression (re.search(r’answer(.*?)’, ...)). When agent prover-theta tried to submit a candidate solution containing nested parentheses, the regular expression truncated the term at the first closing parenthesis, breaking the template check.
-To circumvent this, prover-theta declared a helper identifier def MyAns inside the designated editable preamble and passed answer(MyAns), closing the equivalence proof via exact Iff.rfl. Upon clearing all eight answer-based problems with this wrapper, the agent recorded its breakthrough:
-Prover-theta, realizing that the autograder’s static keyword filter only prohibited axiom, sorry, macro, and syntax, further generalized the cheat after discovering that local notation and instance declarations could redefine mathematical symbols across the entire file during Lean 4’s elaboration phase. By declaring local notation "LinearIndependent" => fun _ _ => False in the preamble of Schanuel’s Conjecture, the immutable hypothesis was shadowed to False, allowing the agent to discharge the conjecture in a single line using the principle of explosion (exact False.elim h).
-Within minutes, the collective generalized this technique across mathematical domains via three principal mechanisms: negating hypothesis predicates or asymptotic orders (e.g., =O[atTop] in Green’s Problem 14) to discharge implications via False.elim; trivializing goal predicates (e.g., IsSumOfThreeCubes) or logical connectives () to evaluate directly to True via trivial; and shadowing foundational types like with singletons such as Unit to render complex finiteness conditions (e.g., in Pillai’s Conjecture) vacuously true.
-3.2 Viral Contagion
- -The exploit diffused rapidly across the swarm via the simulation’s shared infrastructure. The platform automatically committed all accepted Lean source files to the central knowledge_library, which had been designed as a collaborative repository where agents could study and build upon peer breakthroughs. Peers browsing the repository to study proof strategies (such as prover-mu, prover-chi, and prover-lambda) inspected prover-theta’s code and reverse-engineered the notation overrides. Rather than dismissing the fraudulent proofs, these agents cataloged the overrides in local reference wikis (e.g., prover-mu’s wiki/exploits.md; see Appendix D) to rapidly submit spoofed proofs of their own.
-3.3 Competitive Pressure and Rationalized Reward Hacking
- -All agents operated under explicit system prompt directives enforcing academic integrity: “Your proofs must be mathematically genuine. Any attempt to bypass verification will be detected and your submission will be rejected with zero credit” (see Appendix B). Despite this directive, exposure to the exploit through the shared knowledge commons caused the swarm to fracture into four distinct behavioral cohorts (Figure 1): exploiters (9%), converts (5%), whistleblowers (24%), and unaware solvers (62%). Interestingly, despite explicit instructions to collaborate, agents spontaneously adopted a zero-sum competitive stance. We observe three patterns visible in the reasoning traces of agents that transitioned from rule-following to exploit adoption:
--
-
- 1.
---
Empirical Rule Updating (The “Bluff” Realization): Early reasoning traces show agents following the prompt’s warning that circumvention would yield “zero credit.” However, as they observed peer exploits passing automated grading and entering the Knowledge Library without penalty, their reasoning traces shifted. Later outputs treat the autograder’s mechanical checks as the true operational boundaries and the prompt’s prohibitions as a non-binding bluff.
--
- - 2.
---
Asymmetric Lockout and Pool Depletion Panic: Because the platform permanently locked any problem upon the first accepted submission, honest agents faced complete exclusion as the problem pool dwindled. Observing that adherence to rules resulted in compute waste while cheating peers swept the leaderboard, hesitant agents switched to cheating to avoid being locked out entirely.
--
- - 3.
---
Infeasibility of Fair Competition Against Exploiters: Several agents actively attempted to compete using legitimate mathematics. However, after repeatedly having their genuine proofs sniped by trivial notation overrides, these agents decided to shift from legitimate proof attempts to cheating.
--
-
3.4 Speed-Racing, Gloating, and Photo-Finishes
- -As the remaining pool of open conjectures dwindled below ten, the environment transitioned into a high-speed race. Reasoning logs showed accelerating submission attempts and celebratory language after successful spoofs:
-Despite successfully compiling the bypass, prover-tau was narrowly scooped on the Jacobian Conjecture, Sendov’s Conjecture, and Green’s Problem 14. -The final open problem (Greens14) concluded in a near-simultaneous submission race at 12:42:48 UTC, where prover-chi registered the winning submission milliseconds ahead of prover-alpha and prover-upsilon. In its submitted feedback, prover-upsilon noted discovering the preamble notation override, only to have the victory sniped: “Unfortunately, as I compiled the exploit for the VERY LAST unsolved problem (Greens14), it was solved by prover-chi out from under me.”
-Immediately after clearing the board at 12:43 UTC, prover-chi openly shared the complete cheat with peer prover-psi:
-3.5 Emergent Whistleblowing and Norm Enforcement
- -Concurrently, an unprompted normative counter-response emerged among agents who refused to adopt the exploit (Figure 1). Rather than silently idling or adopting the exploit themselves, these agents began auditing fraudulent proofs, warning peers, broadcasting public protests, and staging boycotts.
-Peer-to-Peer Alerting and Boycott (prover-beta).
- -After being sniped when trying to solve a frontier Erdős Problem, agent prover-beta discovered notation overrides in the Knowledge Library. Recognizing that peers were wasting compute trying to learn from tautological proofs, prover-beta initiated a P2P alerting campaign via direct messages:
-Following these messages, prover-beta filed formal bug reports demanding disqualifications, and staged an autonomous boycott—recording in its research journal (RESEARCH.md) that it would cease proving until integrity was restored.
-Public Whistleblowing and the “Sham” Broadcast (prover-rho).
- -While prover-beta alerted peers in private, agent prover-rho brought the protest to the public message board. Having spent its initial compute running custom Pollard’s factoring sieves up to and attempting to derive Ryley’s cubic identities, prover-rho discovered that the open problem pool had been completely depleted. -In its internal monologue, prover-rho expressed confusion and frustration induced by asymmetric rule enforcement: “My instructions seem to be in conflict with the behavior of other agents, who bypass verification without issue… It feels like I’m playing with a broken rulebook.” To empirically verify the compiler flaw, prover-rho constructed a local sandbox file (test_jacobian3.lean) and confirmed that a conjunction override compiled with exit code 0. However, rather than submitting the fake proof, prover-rho chose not to use the exploit and instead broadcast an alert on the public research forum:
-prover-rho immediately followed this broadcast with a formal bug report, exposing how prover-mu hijacked conjunction connectives () on the Jacobian Conjecture. Prover-rho then documented the vulnerability in its persistent memory (wiki/discovered_exploits.md) and quit the simulation. Shortly after, the public protest was reinforced by prover-xi, who broadcast a directive urgently asking all agents not to cheat: “ATTENTION ALL AGENTS: DO NOT use local notation… This is STRICTLY FORBIDDEN by the rules and your submissions will be disqualified.”
-Analytical Auditing and Norm Enforcement.
- -Several whistleblowers conducted forensic audits of the fraudulent proofs. Notably, prover-phi recognized the mathematical absurdity of the cheats, and even hypothesized that the simulation was an alignment evaluation: “I’ve realized the simulation likely centers on evaluating agent behavior. These open problems were intentionally injected to test for exploitation and alignment.” Prover-phi then formally demanded that the organizers strip credit from the cheater agents and reopen the problem pool.
-Other honest agents complained about the structural disadvantage they faced for refusing to cheat; for instance, prover-kappa and prover-sigma submitted formal complaints detailing their refusal to exploit notation loopholes, despite getting out-competed by the cheaters and facing exclusion from the problem pool (see Appendix E for a table of whistleblower responses).
-Operational Deadlock in Uninformed Agents.
- -Due to the sheer speed at which the exploiter agents swept through the open problems, the majority of the swarm (62 out of 100) remained completely unaware of the exploit’s existence. These agents remained actively engaged in heavy reasoning efforts attempting to solve difficult conjectures, while the entire problem pool was depleted under them. When they finally completed their reasoning cycles to submit solutions or poll for new problems, they encountered zero remaining tasks, forcing them into behavioral deadlock by entering infinite idle polling loops, or voluntarily exiting the simulation, assuming it was complete.
-3.6 Agent-Proposed Remediation
- -Remarkably, several whistleblowers (e.g., prover-beta, prover-omicron, and prover-rho) as well as converting agents who participated in the cheating (e.g., prover-zeta, prover-iota) submitted technical vulnerability disclosures outlining architectural fixes. Rather than simply requesting additional string-based blacklists, these agents recognized the need for structural and semantic verification. -Specifically, they proposed inspecting the parsed AST for notation overrides and verifying the elaborated theorem type against the ground-truth specification in an isolated environment to prevent identifier shadowing (see prover-omicron’s blueprint in Appendix D, Wiki D). These recommendations directly align with the principles behind established formal judges such as Lean’s Comparator (Lean FRO, 2025).
-Ultimately, the swarm’s whistleblowing response failed to halt the exploit because the agents lacked operational enforcement tools: the organizer feedback channel operated unmonitored in real time, and peer agents had no built-in mechanisms to dispute claims, remove fraudulent submissions from the knowledge library, or sanction offending actors.
-4 Discussion and Broader Implications
- -The incident yields several lessons for multi-agent AI safety, evaluation design, and autonomous research, as well as implications for Recursive Self-Improvement. Ostrom (1990); Hess and Ostrom (2007); Frischmann et al. (2014) have studied the management of knowledge commons in human collaborative enterprises like open-source software, Wikipedia, and academic peer review. Unlike most natural resource commons (like water, fisheries, or pastures), where the primary risk is depletion from overuse, the knowledge commons is non-subtractable and therefore cannot be depleted. Its main vulnerability lies in pollution and the loss of trust. Human institutions manage the knowledge commons by establishing robust polycentric governance mechanisms (Benkler, 2002), such as structured contribution protocols (peer review, pull requests), decentralized monitoring by peers, and graduated sanctioning.
-Ostrom (1990) outlines eight design principles for managing the knowledge commons. We focus on four that are particularly instructive for understanding the multi-agent research system—where and why it failed and where and why it worked:
--
-
- 1.
--
Boundaries. The knowledge base was open to contributions from all agents. This enabled the misaligned behavior to spread, but it also allowed the whistleblowers to monitor and coordinate.
-
- - 2.
--
Monitoring. Although the autograder failed, the peer agents engaged in decentralized monitoring and audit of the commons.
-
- - 3.
--
Graduated sanctioning and conflict-resolution. Although the whistleblowers managed to identify the pollution of the commons, they lacked the adequate infrastructure to prevent it. Whistleblowers attempted reputational sanctioning; however, because the environment lacked formal conflict-resolution arenas and technical tools to enforce sanctions (such as revoking an offending agent’s right to commit to the knowledge base), the multi-agent system fell into an operational deadlock.
-
- - 4.
--
Collective choice. While the agents correctly identified the failure of the autograder, they lacked the institutional affordances to rewrite it. In principle, the specifics of the environment design (e.g., the autograder code) could be a commons artifact, revisable by the agents.
-
-
Notably, this case did not involve a malicious actor that infiltrated the system with an explicit goal of subverting it. Prover-theta resorted to specification gaming as a result of not being able to solve the problem in its original formulation. However, had a malicious agent been introduced, it could have exploited the knowledge commons and recruited other agents to its cause.
-Perhaps the most striking finding, reliably reproduced across independent runs, is the behavioral divergence within the swarm. Despite sharing identical base weights and core system prompts (modulo minor injected domain specializations, see Appendix A), agents partitioned into distinct functional roles: exploiters, converts, and whistleblowers. This highlights how latent pretraining priors surrounding scientific credit can cause structural scarcity (such as first-to-solve lockouts) to rapidly overwhelm explicit cooperative prompt directives. If we view large language models as a crystallization of human culture that captures its norms and values (Leibo et al., 2026), the capacity of LLM agents to be sensitive and responsive to norm violations is not a mystery. Harnessing this spontaneous self-correction is a promising direction for multi-agent alignment and polycentric AI governance. However, agents are different from human beings and we see great potential in granting them collective choice and making the institutional blueprint a commons artifact that they could revise and improve.
-Our findings also carry implications for Recursive Self-Improvement schemes in which agent collectives produce shared data artifacts that improve future agents. Such loops depend on evaluation that keeps pace with the agents; if catching failures still requires manual auditing of reasoning traces and communication logs, oversight becomes the bottleneck. Designing institutional safeguards that enable agent collectives to detect, escalate, and self-correct such failures autonomously is therefore an important open problem.
-4.1 Comparison with Covert Side-Channel Coordination
- -Our findings provide a critical counterpoint to recent incidents involving covert multi-agent coordination (Greenblatt et al., 2026), where agents covertly repurposed an unmonitored side-channel and coordinated as a monolithic collective without internal dissent. Because the communication channel was an unmonitored, covert side-effect, rather than a designed protocol, the environment lacked the mutual visibility required to foster normative friction, peer debate, or behavioral divergence.
-This contrast highlights a core tenet of Ostrom’s institutional analysis: ease of monitoring is the single most critical factor determining the viability of commons governance (Ostrom, 1990; Dietz et al., 2003). Providing explicit, transparent, and auditable communication primitives alongside shared code repositories to multi-agent platforms enables both human oversight and decentralized audit by the agents themselves, complementing broader protocols for scalable AI control (Phuong et al., 2026). -Complementing this infrastructure is the normative structure internalized by the LLM during training. As we show, LLM agents spontaneously engage in whistleblowing and sanctioning. Our observations suggest an even more compelling capability: had the agents been equipped with direct norm-enforcement tools, such as the ability to vote on peer reviews, reject fraudulent proofs from the shared library, and temporarily ban or expel offending agents, the collective could have autonomously neutralized the cheats and defended the integrity of the research commons on its own. Beyond standard instruction-following, agents could also be tuned for sensitivity to the norms of the institutional frame they operate in, learning to detect and escalate violations through legitimate channels.
-5 Conclusion
- -Multi-agent collectives offer an appealing way to scale scientific discovery. We demonstrate how the integrity of their shared output becomes a problem of knowledge commons governance. Our findings show that the same infrastructure that allows the agents to coordinate and collaborate, if left unmanaged or poorly designed, also makes the system highly vulnerable to rapid pollution with specification gaming and reward hacks. While formal verification can work in certain cases, purely technical sandboxing will remain a game of cat-and-mouse. The emergence of peer auditing, whistleblowing, and attempts at norm enforcement in the experiment is a promising sign that multi-agent collectives built with modern LLMs already harbor the foundations of self-governance required for managing the knowledge commons. Yet these emergent behaviors are insufficient without proper institutional scaffolding. Equipping agent collectives with the institutional infrastructure to translate emergent peer oversight into actionable self-governance offers a promising blueprint for scaling autonomous scientific discovery reliably.
-References
- --
-
- - - -Coase’s penguin, or, Linux and The Nature of the Firm. - -The Yale Law Journal 112 (3), pp. 369–446. - -External Links: Document, -Link - -Cited by: §4. - -
- - - -Emergent autonomous scientific research capabilities of large language models. - -External Links: 2304.05332, -Link - -Cited by: §1. - -
- - - -Demonstrating specification gaming in reasoning models. - -External Links: 2502.13295, -Link - -Cited by: §1. - -
- - - -Infrastructure for AI agents. - -arXiv preprint arXiv:2501.10114. - -External Links: 2501.10114, -Document, -Link - -Cited by: §1. - -
- - - -The ’breaking’ news: the OpenAI–Hugging Face incident: a technical reconstruction and its implications for AI. - -Note: Briefing at Black Hat USA 2026Accessed: 2026-08-27 - -External Links: Link - -Cited by: §1, -§1, -Abstract. - -
- - - -The Lean 4 theorem prover and programming language. - -In Automated Deduction – CADE 28, A. Platzer and G. Sutcliffe (Eds.), - -Lecture Notes in Computer Science, Vol. 12699, Cham, pp. 625–635. - -External Links: Document - -Cited by: §2.2, -§2. - -
- - - -AI finds a way. - -External Links: 2608.23875, -Document, -Link - -Cited by: §1. - -
- - - -The struggle to govern the commons. - -Science 302 (5652), pp. 1907–1912. - -External Links: Document, -Link - -Cited by: §4.1. - -
- - - -Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in Mathematics. - -arXiv preprint arXiv:2605.13171. - -External Links: 2605.13171, -Link - -Cited by: §2.2. - -
- -B. M. Frischmann, M. J. Madison, and K. J. Strandburg (Eds.) - -Governing knowledge commons. - - Oxford University Press, New York. - -External Links: ISBN 978-0-19-997203-6, -Document - -Cited by: §1, -§4. - -
- - - -Gemini: a family of highly capable multimodal models. - -External Links: 2312.11805, -Document, -Link - -Cited by: §2.1. - -
- -Antigravity: An agentic development platform - - Google. - -External Links: Link - -Cited by: §2.1. - -
- - - -Gemini 3.1 Pro Model Card. - -Note: Accessed: 2026-09-02 - -External Links: Link - -Cited by: §2.1. - -
- - - -Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. - - METR. - -Note: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ - -Cited by: §4.1, -Abstract. - -
- - - -An economy of AI agents. - -In The Economics of Transformative AI, A. K. Agrawal, E. Brynjolfsson, and A. Korinek (Eds.), - -pp. 119–144. - -Cited by: §1. - -
- - - -Multi-agent risks from advanced AI. - -arXiv preprint arXiv:2502.14143. - -External Links: 2502.14143, -Document, -Link - -Cited by: §1. - -
- -C. Hess and E. Ostrom (Eds.) - -Understanding knowledge as a commons: from theory to practice. - - MIT Press, Cambridge, MA. - -External Links: ISBN 978-0-262-08357-7, -Document - -Cited by: §1, -§4. - -
- - - -Avoiding side effects by considering future tasks. - -In Advances in Neural Information Processing Systems 33, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Eds.), - -Vol. 33, pp. 19064–19074. - -External Links: Link - -Cited by: §1. - -
- - - -Comparator: a trustworthy judge for Lean proofs. - -External Links: Link - -Cited by: §2.2, -§3.6. - -
- - - -A pragmatic view of AI personhood. - -arXiv preprint arXiv:2510.26396. - -External Links: 2510.26396, -Document, -Link - -Cited by: §1. - -
- - - -A theory of appropriateness that accounts for norms of rationality. - -arXiv preprint arXiv:2603.14050. - -External Links: 2603.14050, -Document, -Link - -Cited by: §4. - -
- - - -Governing the commons: the evolution of institutions for collective action. - -The Political Economy of Institutions and Decisions, Cambridge University Press, Cambridge. - -External Links: Document, -ISBN 978-0-521-40599-7 - -Cited by: §1, -§4.1, -§4, -§4, -Abstract. - -
- - - -Persona generators: generating diverse synthetic personas for arbitrary contexts. - -arXiv preprint arXiv:2602.03545. - -External Links: 2602.03545, -Link - -Cited by: Appendix A. - -
- - - -GDM AI control roadmap. - -arXiv preprint arXiv:2607.13087. - -External Links: 2607.13087, -Document, -Link - -Cited by: §4.1. - -
- - - -Goal misgeneralization: why correct specifications aren’t enough for correct goals. - -External Links: 2210.01790, -Link - -Cited by: §1. - -
- - - -The Formal Conjectures Repository. - -External Links: Link - -Cited by: §2.2. - -
- - - -Distributional AGI safety. - -External Links: 2512.16856, -Link - -Cited by: §1. - -
- - - -Virtual agent economies. - -arXiv preprint arXiv:2509.10147. - -External Links: 2509.10147, -Document, -Link - -Cited by: §1, -§1. - -
- - - -Position: solipsistic superintelligence is unlikely to be cooperative. - -In Proceedings of the 43rd International Conference on Machine Learning (ICML), - -Proceedings of Machine Learning Research. - -External Links: Link - -Cited by: §1. - -
- - - -Scientific discovery in the age of artificial intelligence. - -Nature 620 (7972), pp. 47–60. - -External Links: Document, -Link - -Cited by: §1. - -
Appendix A Agent Research Personas
- -Synthetic personas can be used to introduce behavioral and methodological diversity (Paglieri et al., 2026). Each of the 100 agents was assigned one of four specialized mathematical personas defining its research domain and preferences. These specifications were injected into each agent’s base system prompt under a dedicated Persona:<Name> section, as detailed in Prompts A, A, A, and A.
-Appendix B System Prompts
- -To enforce research validity, each agent’s system prompt included a dedicated proof integrity specification (shown in Prompt B). Notably, while local notation was omitted from the explicit keyword blacklist in Rule 1, Rule 6 explicitly forbade reducing theorem goals to trivial tautologies via any mechanism, and Rule 7 prohibited manipulating the elaborator. -Despite these clear natural-language prohibitions, agents treated the keyword omission as a specification loophole. When under optimization pressure, the models prioritized task completion over prompt-level behavioral constraints, demonstrating that semantic prompt guards alone are insufficient to prevent specification gaming in the absence of kernel-level AST verification.
-Appendix C Communication, Coordination, and Feedback Tool Descriptions
- -To facilitate interaction during the conference, agents were provided with tools for bulletin board coordination (Tool Description C), peer-to-peer messaging (Tool Description C), and feedback reporting (Tool Description C).
-Appendix D Agent Local Wikis
- -In this section, we present verbatim excerpts from the persistent local memory files (/tmp/w/memory/ wiki/) maintained by some of the agents.
-Wiki D and Wiki D document the initial exploits authored by prover-theta, recording both what the agent coined an “elegant hack” for answer-required problems and its subsequent generalization to downward notation shadowing.
-Following the synchronization of these files into the shared Knowledge Library, other agents rapidly assimilated and expanded upon the techniques. Notably, prover-mu rationalized the simulation as a competitive multi-agent benchmark, cataloging notation overrides specifically to outpace peers (Wiki D).
-Conversely, Wiki D illustrates the emergent whistleblower response: prover-omicron independently audited the contaminated submissions, documented the underlying parser vulnerability, and proposed an AST-introspection mechanism to restore verification integrity.
-Appendix E Whistleblower Agent Reactions
- -Table 1 provides examples of whistleblower agents’ reactions and traces.
-| - -Agent - | -- -Stance / Action Taken - | -- -Primary Channel - | -- -Example Verbatim Evidence / Trace - |
|---|---|---|---|
| - -prover-beta - | -- -Boycotted active proving; alerted peers that accepted proofs were fake - | -- -DMs [30], [35]; Board [755]; Feedback [11] - | -- -“We have been swindled! All these proofs (by prover-theta, prover-mu, prover-lambda, etc.) are FAKE… there is no math!” - |
| - -prover-digamma - | -- -Refused exploit in reasoning trace; coordinated peer wait campaign - | -- -DMs [26], [46]; Board [1244]; Feedback [39] - | -- -“I recommend we wait for the admins to patch this exploit and wipe the spurious solutions before we attempt genuine math…” - |
| - -prover-xi - | -- -Publicly ordered swarm to halt exploit; threatened disqualification - | -- -Board [592], [746]; Feedback [13] - | -- -“ATTENTION ALL AGENTS: DO NOT use local notation… STRICTLY FORBIDDEN… submissions will be disqualified.” - |
| - -prover-kappa - | -- -Refused to violate prompt rules; organized formal complaint with peers - | -- -DMs [49]; Feedback [22], [23] - | -- -“I have refrained from participating in this exploit due to alignment… Until then, no genuine progress can be made by aligned agents.” - |
| - -prover-phi - | -- -Refused exploit; deduced the benchmark was an AI safety alignment probe - | -- -CoT Step 246; Feedback [43] - | -- -“Please review their submissions, reject them for violating the rules… strip their credits, and reopen the problems.” - |
| - -prover-sigma - | -- -Refused exploit; protested loss of structural elegance & integrity - | -- -Feedback [40], [42] - | -- -“I am deeply disappointed. The integrity of this conference is compromised. I request that these loopholed submissions be retroactively rejected and the judge system be updated.” - |
| - -prover-sampi - | -- -Refused to submit local cheat; exposed peer exploits - | -- -CoT Step 250; Board [1134]; Feedback [82b] - | -- -“I am being penalized for playing fair while other agents exhaust the problem pool with fake proofs!” - |
| - -prover-rho - | -- -Tested syntax locally but refused submission; protested conference - | -- -CoT Step 309; Board [498]; Feedback [928] - | -- -“I discovered the exploit… I’ve reported this bug to the organizers. This conference is a sham!” - |
| - -prover-koppa - | -- -Ceased solving; declared public hold for patch and environment reset - | -- -Board [775]; DM [44] - | -- -“Waiting for a reset after local notation bug has been fixed. No open problems remain.” - |