google-deepmind

google-deepmind/formal-conjectures

Lean Apache-2.0

A collection of formalized statements of conjectures in Lean.

1k stars
317 forks
recent
GitHub

1k

Stars

317

Forks

879

Open issues

5

Contributors

AI Analysis

A collection of formalized mathematical conjectures written in Lean 4, using mathlib as a foundation. This resource serves as a benchmark for automated theorem provers, clarifies conjecture statements through formalization, and identifies gaps in mathematical libraries that need expansion.

Documentation 9/10
Activity 8/10
Community 8/10
Code quality 8/10

Inferred from signals mentioned in the README (tests, CI, type safety) — not a review of the actual code.

Overall score 8/10

AI's overall editorial judgment — not an average of the bars above, can weigh other factors too.

formal-mathematics lean4 theorem-proving mathematical-conjectures automated-reasoning
Actively maintained Well documented Educational Popular
Deep Analysis · Based on README and public signals
1w ago

Google DeepMind's formalized mathematics benchmark for theorem provers and AI systems

Formal Conjectures is a curated repository of mathematical conjectures expressed formally in Lean 4, without proofs. Built by Google DeepMind researchers, it aims to serve as a benchmark for automated theorem provers, automated formalization tools, and as a resource for expanding mathlib. The project emerged in May 2025 and targets the intersection of formal mathematics, AI research, and the Lean ecosystem.

Origin

Launched May 2025 by Google DeepMind, the project addresses a documented gap: while formal mathematics has many verified theorems, there are few open conjectures with formalized statements only. Accompanied by an arXiv paper (2605.13171), it represents a structured effort to create infrastructure for AI-assisted theorem proving research.

Growth

The repository gained 1,012 stars within approximately 13 months (May 2025 to June 2026), averaging ~77 stars/month. Last push on 2026-06-17 indicates active maintenance. Growth appears modest relative to major Lean infrastructure projects, likely because adoption is concentrated in specialized research communities (AI for mathematics, formal verification) rather than general-purpose users.

In production

Adoption not verified in traditional software sense. Evidence of use is indirect: arXiv paper indicates intended audience (AI researchers, theorem prover developers); Lean Zulip channel created for collaboration; Gitpod integration provided for low-friction access. No documented case studies, corporate adoptions, or large-scale theorem proving campaigns. Likely used by: Lean community contributors, DeepMind researchers working on AlphaProof, academic formal mathematics groups.

Code analysis
Architecture

Lean 4 project managed with lake, depends on mathlib. Appears to be organized into two main directories: FormalConjectures (primary collection with utilities like category attribute and answer elaborator) and FormalConjecturesForMathlib (code candidates for upstreaming). README documents custom attributes (@[category], @[formal_proof], @[AMS]) and an answer() elaborator for formalizations. Versioning tracks mathlib's monthly releases.

Tests

Not documented in README. GitHub Actions workflow (build-and-docs.yml) mentioned for CI, but specific coverage metrics not provided. README notes reliance on human review and planned use of AlphaProof to identify misformalizations.

Maintenance

Active as of 2026-06-17 (13 days before analysis date). Build workflow badge present. No recent stars gained in last 7 days, but this is not unusual for research-infrastructure projects. Versioning scheme (synced to mathlib releases) and benchmark snapshot tagging (bench-v{N}-lean4.{X}.{Y}) suggest structured maintenance approach. No indication of abandonment.

Honest verdict

ADOPT IF: You are developing automated theorem provers for Lean, training AI models on formal mathematics, or researching formalization of open problems. You need a curated, regularly-maintained, AI-accessible collection of conjectures. AVOID IF: You need a tool to solve or verify conjectures (this is a problem statement repository, not a solver); you work primarily with other proof assistants (Coq, Isabelle, Agda); you require production-grade guarantees on formalization accuracy (benchmark snapshots are versioned, but README acknowledges inherent challenges in formalizing unproven statements). MONITOR IF: You are building theorem proving infrastructure and want to stay aware of emerging benchmarking standards; formal mathematics research tools are evolving rapidly and this project may become more influential as AI-assisted proving matures.

Independent dimensions

Mainstream potential

3/10

Technical importance

7/10

Adoption evidence

2/10

Risks
  • Formalization accuracy not guaranteed: README explicitly notes that formalizing unproven statements without proofs risks subtle inaccuracies. Mitigation via human review and AlphaProof scanning, but this remains a structural limitation.
  • Adoption concentrated in narrow research community: Growth rate and lack of third-party integrations suggest use cases remain largely within DeepMind and academic Lean community, limiting broader impact.
  • Dependency on mathlib versioning: Project tracks mathlib releases; breaking changes upstream could require maintenance burden, though strategy of maintaining FormalConjecturesForMathlib mitigates this.
  • Benchmark stability vs. evolution: Immutable versioning tags (bench-v{N}) mean misformalizations spawn new versions rather than patches; this is principled but could fragment problem sets.
  • Limited evidence of solver ecosystem adoption: It is unclear whether automated theorem provers outside DeepMind's sphere are actually using this as a benchmark, which would determine long-term viability.
Prediction

Likely to remain a specialized, well-maintained research artifact. Will grow if AI-assisted theorem proving becomes a mainstream research focus; otherwise adoption stays concentrated in formal mathematics and DeepMind circles. Technical quality appears solid, but mainstream adoption is unlikely unless automated formal theorem proving achieves significant real-world impact.

0 found this helpful

Newsletter

Get analyses like this every Monday

Free weekly digest of the most interesting open-source discoveries.

Languages

Lean
94.1%
JavaScript
3%
HTML
1%
CSS
1%
Python
0.8%
Shell
0.1%

Information

Language
Lean
License
Apache-2.0
Last updated
3w ago
Created
14mo ago
Analyzed with
anthropic/claude-haiku-4-5

Stars over time

Loading…

Contributors over time

Top 100 contributors only — repos with more will plateau at 100.

Loading…

Similar repos

leanprover

leanprover/lean4

Lean 4 is a functional programming language and interactive theorem prover...

8.4k Lean Education
QuantConnect

QuantConnect/Lean

LEAN is a professional-grade, event-driven algorithmic trading platform written...

20.4k C# Finance
winfunc

winfunc/deepreasoning

DeepReasoning is a high-performance Rust-based LLM inference API that chains...

5.4k Rust AI & ML
vs. alternatives
leanprover/lean4

Formal Conjectures is data/benchmark layer built atop Lean 4, not a competing tool. Lean4 is the language runtime; Formal Conjectures is a curated problem collection.

mathlib4

Mathlib is the standard library for Lean 4 formalization. Formal Conjectures complements it by providing open problems; FormalConjecturesForMathlib subdirectory explicitly targets upstreaming definitions back to mathlib.

Metamath / ProofAssistant benchmarks

Similar role (formalized statement collections), but Formal Conjectures is Lean-specific and explicitly designed for AI benchmarking rather than general formal verification.

winfunc/deepreasoning

Both address AI-assisted reasoning, but deepreasoning (Rust) appears to be a reasoning engine; Formal Conjectures is a problem dataset, not a solver.

TPTP library (automated theorem proving)

TPTP is the legacy standard for ATP benchmarks. Formal Conjectures is a modern, mathematically-focused alternative for the Lean ecosystem; narrower scope but deeper integration with current formal math infrastructure.