Primary source: https://arxiv.org/pdf/2601.21165v1 Verified: 2026-10-12 Extraction: pdftotext -layout; page numbers are one-based PDF pages. --- PDF page 6 --- Figure 4: Each task in the research set is graded using a rubric totaling 10 points that can be used by an expert or a judge model. To scale our ability to evaluate models, we use another model to judge responses. 5.2 at “xhigh”, and without browsing. In our initial evaluations, GPT-5.2 is the top performing model on FrontierScience, scoring 77% on the Olympiad Set and 25% on the Research set. Surprisingly, GPT-5 outperforms GPT-5.1 on the Research set and ties GPT-5.2. Overall, we’ve seen substantial progress on solving expert-level questions while leaving headroom for more progress, especially on open-ended research-style tasks. For both sets, we use a model-based judge of GPT-5 (with a reasoning effort of ”high”) to evaluate the models. For Olympiad, we give the judge the attempted answer and the actual answer and ask it to compare equivalency of the expression, number, or phrase. For Research, we give the judge the attempted answer and the rubric and ask it to return a single number reflecting the number of rubric points the answer earned (Appendix B). When separating by subject, models perform comparably across distributions. For the Olympiad set, models perform better on chemistry, followed by physics and biology. For the Research set, models perform better on chemistry, followed by biology and then physics. Analyzing the transcripts, models typically struggled with reasoning or logic error, failures in understanding niche concepts, calculation errors, and factual inaccuracy. 6 --- PDF page 7 --- Figure 5: The Olympiad split is composed of a diverse set of topics, from biochemistry to quantum mechanics. Figure 6: We compare accuracies across several frontier models. GPT-5.2 is our highest performing model across the Olympiad and the Research set. Gemini 3 Pro is comparable to GPT-5.2 on Olympiad, and GPT-5 is tied with GPT-5.2 on Research. For all Olympiad evaluations, scores were averaged across 20 independent trials. For all Research evaluations, scores were averaged across 30 independent trials, using a threshold of a response earning at least seven rubric points as correct. 4 D ISCUSSION While FrontierScience represents an advance in understanding scientific research capabilities, there are multiple limitations: 1. Constrained problem-solving: A significant part of scientific research is proposing novel research directions, hypotheses, and ideas. FrontierScience is composed of questions with a constrained problem statement, which focuses on evaluating the reasoning to complete a research task and less on ideation. While the Research set aims to measure more open-ended reasoning, this is an inherent limitation of an autogradable Q&A style evaluation. 2. Rubric reliability: We sought to improve rubric reliability of the Research set through strict guidelines, verification, and consistency with human grading. However, the rubric is less objective than the equivalency checker of a single expression or number, and relies on the model judge’s capabilities. 3. Modalities: Problems are designed to be text-only without image or video outputs. Modal- ities beyond text are more representative of scientific research. In particular, real-world scientific research often involves interaction with reality (e.g., wet labs), which this evalua- tion does not cover. 7 --- PDF page 8 --- Figure 7: We compare accuracies across several frontier models on FrontierScience-Olympiad (Left) and FrontierScience-Research (Right) with accuracies split out by subject. Figure 8: We compare accuracies for GPT-5.2 and OpenAI o3 on FrontierScience-Olympiad and FrontierScience-Research across different reasoning efforts. Using more test-time tokens enables GPT-5.2 to go from 67.5% to 77.1% on the Olympiad set, as well as from 18% to 25% on the Research set. Surprisingly, o3 performs marginally worse at high reasoning effort compared to medium reasoning effort on the Research set. 4. Human baselining: We did not perform human baselining of this dataset and leave that to future work. Since the questions are grounded in experts’ authentic research, an interesting question is how to conduct a human baseline. Since the questions are so specialized, it may be important to find experts in that speciality to solve them and derive a consensus baseline. Research and practical evaluations will be important to continue building long standing and directly relevant evaluations. Scientific reasoning is important for the beneficial impacts of AI and we hope for continued development of robust and relevant benchmarks for accelerating scientific progress. 5 R ELATED W ORK Previous science-oriented and knowledge benchmarks primarily measure models’ capabilities through multiple-choice or single-answer formats. Benchmarks such as MMLU (Hendrycks et al., 2021; Wang et al., 2024b), GPQA (Rein et al., 2023), and ScienceQA (Lu et al., 2022) have significantly contributed to understanding model performance across scientific knowledge and basic reasoning tasks. However, these benchmarks largely target knowledge retrieval or recognition of well-known scientific concepts rather than research-level scientific reasoning. GPQA, for instance, measures general-purpose science reasoning but is limited to structured multiple-choice settings, reducing diagnostic power of more complex, open-ended tasks. 8