Mistral Releases Leanstral 1.5: A Breakthrough in Formal Math and Code Verification

Mistral AI has officially launched Leanstral 1.5, a powerful open-source model specifically engineered for formal verification using the Lean 4 programming language. By mastering the rigors of mathematical proofs and software correctness, this model marks a significant leap forward in the utility of open-source AI for high-stakes technical environments.

Dominating Formal Mathematics Benchmarks

Leanstral 1.5 is designed to navigate the complex logic required by Lean 4, a language built for verifying mathematical proofs and software reliability. The model's performance on specialized benchmarks is nothing short of extraordinary. It achieved a perfect 100 percent score on miniF2F, a benchmark ranging from high school level mathematics to the intense difficulty of math olympiads.

When tested against the PutnamBench—a collection of 672 problems from the prestigious Putnam math competition—Leanstral 1.5 successfully solved 587 problems. This performance places it at the top of the open-source field, trailing only the closed-source Aleph Prover. Furthermore, the model demonstrated mastery over graduate-level mathematics in algebra, scoring 87 percent on the FATE-H benchmark and 34 percent on FATE-X, which cover advanced topics like group and ring theory.

Beyond Math: Real-World Code Bug Detection

While its mathematical prowess is the headline, Leanstral 1.5 proves to be a formidable tool for software engineers through its code verification capabilities. Mistral demonstrated the model's practical utility by deploying it to scan 57 different open-source repositories.

In this real-world application, the model successfully identified five previously unknown bugs. One notable discovery was an overflow bug located within the varinteger Rust library. This ability to catch subtle, high-consequence errors in production-level code suggests that Leanstral 1.5 can serve as an automated "sanity check" for developers working in memory-safe or mission-critical languages.

Technical Rigor and Accessibility

The development of Leanstral 1.5 involved a sophisticated training pipeline consisting of mid-training, supervised fine-tuning (SFT), and reinforcement learning (RL). This multi-stage approach ensures that the model does not just predict the next token but understands the underlying logical structures required for formal reasoning.

In a move that strengthens the open-source ecosystem, Mistral has released the model under the Apache 2.0 license. This allows developers and researchers to integrate, modify, and deploy the model with minimal restrictions. Leanstral 1.5 is currently available through Hugging Face and via a free API, lowering the barrier to entry for teams looking to implement formal verification in their workflows.

Key Takeaways

  • Benchmark Excellence: Leanstral 1.5 achieved a 100% score on miniF2F and outperforms almost all open-source competitors on PutnamBench and graduate-level algebra tests.
  • Practical Debugging: The model has proven its ability to find real-world vulnerabilities, such as an overflow bug in a Rust library, during repository scans.
  • Open-Source Empowerment: Released under the Apache 2.0 license, the model is highly accessible via Hugging Face and a free API for the global developer community.