google-deepmind/formal-conjectures
Lean Apache-2.0A collection of formalized statements of conjectures in Lean.
1k
Stars
317
Forks
879
Open issues
5
Contributors
AI Analysis
A collection of formalized mathematical conjectures written in Lean 4, using mathlib as a foundation. This resource serves as a benchmark for automated theorem provers, clarifies conjecture statements through formalization, and identifies gaps in mathematical libraries that need expansion.
Inferred from signals mentioned in the README (tests, CI, type safety) — not a review of the actual code.
AI's overall editorial judgment — not an average of the bars above, can weigh other factors too.
Google DeepMind's formalized mathematics benchmark for theorem provers and AI systems
Formal Conjectures is a curated repository of mathematical conjectures expressed formally in Lean 4, without proofs. Built by Google DeepMind researchers, it aims to serve as a benchmark for automated theorem provers, automated formalization tools, and as a resource for expanding mathlib. The project emerged in May 2025 and targets the intersection of formal mathematics, AI research, and the Lean ecosystem.
Launched May 2025 by Google DeepMind, the project addresses a documented gap: while formal mathematics has many verified theorems, there are few open conjectures with formalized statements only. Accompanied by an arXiv paper (2605.13171), it represents a structured effort to create infrastructure for AI-assisted theorem proving research.
The repository gained 1,012 stars within approximately 13 months (May 2025 to June 2026), averaging ~77 stars/month. Last push on 2026-06-17 indicates active maintenance. Growth appears modest relative to major Lean infrastructure projects, likely because adoption is concentrated in specialized research communities (AI for mathematics, formal verification) rather than general-purpose users.
Adoption not verified in traditional software sense. Evidence of use is indirect: arXiv paper indicates intended audience (AI researchers, theorem prover developers); Lean Zulip channel created for collaboration; Gitpod integration provided for low-friction access. No documented case studies, corporate adoptions, or large-scale theorem proving campaigns. Likely used by: Lean community contributors, DeepMind researchers working on AlphaProof, academic formal mathematics groups.
Lean 4 project managed with lake, depends on mathlib. Appears to be organized into two main directories: FormalConjectures (primary collection with utilities like category attribute and answer elaborator) and FormalConjecturesForMathlib (code candidates for upstreaming). README documents custom attributes (@[category], @[formal_proof], @[AMS]) and an answer() elaborator for formalizations. Versioning tracks mathlib's monthly releases.
Not documented in README. GitHub Actions workflow (build-and-docs.yml) mentioned for CI, but specific coverage metrics not provided. README notes reliance on human review and planned use of AlphaProof to identify misformalizations.
Active as of 2026-06-17 (13 days before analysis date). Build workflow badge present. No recent stars gained in last 7 days, but this is not unusual for research-infrastructure projects. Versioning scheme (synced to mathlib releases) and benchmark snapshot tagging (bench-v{N}-lean4.{X}.{Y}) suggest structured maintenance approach. No indication of abandonment.
ADOPT IF: You are developing automated theorem provers for Lean, training AI models on formal mathematics, or researching formalization of open problems. You need a curated, regularly-maintained, AI-accessible collection of conjectures. AVOID IF: You need a tool to solve or verify conjectures (this is a problem statement repository, not a solver); you work primarily with other proof assistants (Coq, Isabelle, Agda); you require production-grade guarantees on formalization accuracy (benchmark snapshots are versioned, but README acknowledges inherent challenges in formalizing unproven statements). MONITOR IF: You are building theorem proving infrastructure and want to stay aware of emerging benchmarking standards; formal mathematics research tools are evolving rapidly and this project may become more influential as AI-assisted proving matures.
Independent dimensions
Mainstream potential
3/10
Technical importance
7/10
Adoption evidence
2/10
- Formalization accuracy not guaranteed: README explicitly notes that formalizing unproven statements without proofs risks subtle inaccuracies. Mitigation via human review and AlphaProof scanning, but this remains a structural limitation.
- Adoption concentrated in narrow research community: Growth rate and lack of third-party integrations suggest use cases remain largely within DeepMind and academic Lean community, limiting broader impact.
- Dependency on mathlib versioning: Project tracks mathlib releases; breaking changes upstream could require maintenance burden, though strategy of maintaining FormalConjecturesForMathlib mitigates this.
- Benchmark stability vs. evolution: Immutable versioning tags (bench-v{N}) mean misformalizations spawn new versions rather than patches; this is principled but could fragment problem sets.
- Limited evidence of solver ecosystem adoption: It is unclear whether automated theorem provers outside DeepMind's sphere are actually using this as a benchmark, which would determine long-term viability.
Likely to remain a specialized, well-maintained research artifact. Will grow if AI-assisted theorem proving becomes a mainstream research focus; otherwise adoption stays concentrated in formal mathematics and DeepMind circles. Technical quality appears solid, but mainstream adoption is unlikely unless automated formal theorem proving achieves significant real-world impact.
Explore similar
Newsletter
Get analyses like this every Monday
Free weekly digest of the most interesting open-source discoveries.
Languages
Information
- Language
- Lean
- License
- Apache-2.0
- Last updated
- 3w ago
- Created
- 14mo ago
- Analyzed with
- anthropic/claude-haiku-4-5
Stars over time
Contributors over time
Top 100 contributors only — repos with more will plateau at 100.
Open issues
Integer factorization in P?
Autoformalization error on `auto_oeis`: `a089026` in `7406_a8289e8e.lean` encodes A000188, not A089026 (statement false at n=4)
Add IMO 2013 Problem 6 (beautiful labellings count) as a research-solved combinatorics statement
Top contributors
Recent releases
Similar repos
QuantConnect/Lean
LEAN is a professional-grade, event-driven algorithmic trading platform written...
| Repository | Stars | Week Δ | Language | Score | Updated |
|---|---|---|---|---|---|
|
|
1k | — | Lean | 8/10 | 3w ago |
|
|
8.4k | — | Lean | 8/10 | 13h ago |
|
|
20.4k | — | C# | 8/10 | 2d ago |
|
|
5.4k | — | Rust | 7/10 | 9mo ago |
Formal Conjectures is data/benchmark layer built atop Lean 4, not a competing tool. Lean4 is the language runtime; Formal Conjectures is a curated problem collection.
Mathlib is the standard library for Lean 4 formalization. Formal Conjectures complements it by providing open problems; FormalConjecturesForMathlib subdirectory explicitly targets upstreaming definitions back to mathlib.
Similar role (formalized statement collections), but Formal Conjectures is Lean-specific and explicitly designed for AI benchmarking rather than general formal verification.
Both address AI-assisted reasoning, but deepreasoning (Rust) appears to be a reasoning engine; Formal Conjectures is a problem dataset, not a solver.
TPTP is the legacy standard for ATP benchmarks. Formal Conjectures is a modern, mathematically-focused alternative for the Lean ecosystem; narrower scope but deeper integration with current formal math infrastructure.