Researchers from Tsinghua University, Moonshot AI and the Shanghai Qizhi Institute have released TCSExam, a benchmark built to test whether AI agents can do theoretical computer science research rather than answer textbook questions about it. The MoonshotAI/TCSExam repository went public on GitHub on November 3, 2026 under the Apache-2.0 licence, and the paper, "TCSExam: Evaluating Frontier LLMs on the Full Range of TCS Research", followed on arXiv (2611.07962) on November 5. Its five authors are Yifan Luo, Zhengying Liu, Yang Yuan, Zhilin Yang and the computer scientist Andrew Chi-Chih Yao.
The Full suite contains 100 expert-reviewed tasks split into four tracks. Twenty are theorems taken from recent TCS papers (2023 to 2026) that must be proved in Lean 4. Thirty are solver-engineering problems drawn from competitions and instance libraries, where the agent submits a reusable program that is judged on hidden instances. Twenty are constructions such as codes, graphs and packings, scored against documented records. The last thirty are open conjectures, including log-rank and Ryser, posed in prove-or-refute form. Every submission is checked by the Lean kernel or an executable checker, so no score depends on an LLM grader. A 20-task Lite subset keeps five tasks per track and cuts the number of attempts by 80% at the same 12-hour budget per attempt.
Eight agents were evaluated on Full. GPT-6 Astra, run through Codex, leads the TCSExam Index at 71.9, ahead of Kimi K3.1 Pro at 57.4 and Kimi K3 at 46.8; DeepSeek V4.1 Flash 1017, GLM-5.3, Qwen3.8 Max and Grok 4.6 land between roughly 40 and 46. Astra completed 14 of 20 theorems without any supplied intermediate lemmas, a 70% success rate, while no other agent got past 38.3%. On Lite, where Anthropic's Opus 5.5 was added, Opus comes first at 72.3 against 67.0 for Astra. The authors report that the two suites order the eight shared agents almost identically, with a Spearman correlation of 0.93.
The open-conjecture column is the most sobering part. No agent produced an original, complete resolution of any of the 30 conjectures. The paper is explicit about the two kinds of accepted submissions: Kimi K3.1 Pro and DeepSeek V4.1 Flash 1017, which ran with internet access, retrieved counterexamples and Lean proofs that OpenAI had already published for Ryser's conjecture and the modified integer round-up conjecture, then adapted them to the task definitions. The tasks had been written before OpenAI released those results.
Narrower verified mathematics did come out of the runs. GPT-6 Astra proved the Alon-Jaeger-Tarsi conjecture over the field with five elements for every dimension up to 9, extending earlier results that stopped at 7 and 8, and the authors say an independent Lean check confirms it using only standard axioms. In the construction track, Kimi K3 built a 124-vertex graph with 630 edges and no 3- or 4-cycles, one edge above the task reference, plus a [[25,7,6]] quantum stabilizer code that lifts the reference distance from 5 to 6. Astra also showed that binary codes of length 22 and minimum distance 10 have at most 82 codewords, tightening the published bound of 84.
For people who build or evaluate agents, the release is practical as well as academic. The repository ships task sources, Docker build recipes and evaluation tools in the Harbor format, uses Kimi Code as its default harness and documents settings for Claude Code, Codex and Grok Build. Because every result passes a formal or executable check, the benchmark separates genuine proofs from convincing prose, something many reasoning evaluations struggle to do. Independent outlets including On The Wire, Aivio News and Crypto Briefing have covered the paper, mostly focusing on the gap between strong theorem proving and the empty conjecture column.
Several caveats apply. The scores come from the benchmark team and the paper is a preprint without peer review. Two authors work at Moonshot AI, whose Kimi models rank second and third on Full. The theorem set was screened in pilot runs, with more than 40 problems retired after being solved, which the authors acknowledge can skew comparisons with the screening agent. Training exposure to public solver instances is unknown, Opus 5.5 was only run on Lite, and each number reflects a complete agent (model, harness and tool access) rather than a bare model.