RanXinByteDance commited on
Commit
8f19cb1
·
verified ·
1 Parent(s): e396b93

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -67
README.md CHANGED
@@ -1,94 +1,64 @@
1
- # BFS-Prover
2
 
3
- BFS-Prover is a scalable automatic theorem prover that leverages Best-First Tree Search (BFS) to navigate proof search spaces efficiently. This model achieves state-of-the-art performance on the MiniF2F test benchmark with a score of 72.95%, demonstrating that properly scaled BFS can match or exceed the performance of more complex search methods.
4
 
5
  ## Model Details
6
 
7
- - **Architecture**: Based on Qwen2.5-Math-7B
8
- - **Task**: Automatic theorem proving in Lean4
9
- - **Training**: Trained through expert iteration with SFT and DPO
10
- - **License**: apache-2.0
11
- - **Framework**: LeanDojo for Lean4 integration
12
-
13
- ## Key Features
14
-
15
- 1. **Expert Iteration with Self-Filtering**
16
- - Strategic filtering of problems solvable by beam search
17
- - Progressive focusing on harder theorems
18
- - Continuous policy improvement through iterative training
19
-
20
- 2. **Direct Preference Optimization (DPO)**
21
- - Leverages compiler feedback for policy refinement
22
- - Uses positive and negative tactic pairs for learning
23
- - Improves sampling efficiency during proof search
24
-
25
- 3. **Length-Normalized BFS**
26
- - Incorporates path length normalization
27
- - Enables effective exploration of deeper proof paths
28
- - Balances between shallow and deep reasoning
29
 
30
  ## Performance
31
 
32
- - **MiniF2F Test Score**: 72.95% (accumulative)
33
- - **Single Run Score**: 70.83% ± 0.89%
34
- - **Search Configuration**:
35
- - Temperature: 1.1
36
- - Expansion width: 2
37
- - Length normalization factor: 0.5
38
 
39
  ## Usage
40
 
41
  ```python
 
42
  from transformers import AutoModelForCausalLM, AutoTokenizer
43
 
44
- # Load tokenizer and model
45
- model_name = "bytedance-research/BFS-Prover"
46
- tokenizer = AutoTokenizer.from_pretrained(model_name)
47
- model = AutoModelForCausalLM.from_pretrained(model_name)
48
-
49
- # For memory-efficient loading
50
- model = AutoModelForCausalLM.from_pretrained(
51
- model_name,
52
- device_map="auto", # Automatic device mapping
53
- load_in_8bit=True # Or load_in_4bit=True for more memory savings
54
- )
55
- ```
56
 
57
- ## Required Environment
58
-
59
- - Python 3.8+
60
- - Lean4
61
- - LeanDojo
62
- - transformers
63
- - torch
64
 
65
- ## Limitations
66
 
67
- - Based on a 7B parameter model, which may limit capture of complex mathematical patterns
68
- - Context window constraints may affect handling of extensive proof states
69
- - Trade-off between model size and inference speed in tree search
70
 
71
  ## Citation
72
 
 
 
73
  ```bibtex
74
- @article{bfs-prover2024,
75
  title={BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Proving},
76
  author={Xin, Ran and Xi, Chenguang and Yang, Jie and Chen, Feng and Wu, Hang and Xiao, Xia and Sun, Yifan and Zheng, Shen and Shen, Kai},
77
- year={2024}
 
78
  }
79
  ```
80
 
81
- ## Contributors
 
 
82
 
83
- Key Contributors:
84
- - Ran Xin (Seed Foundation Code, ByteDance)
85
- - Chenguang Xi (Seed Foundation Code, ByteDance)
86
- - Jie Yang (Applied Machine Learning, ByteDance)
87
- - Feng Chen (Stanford University)
88
 
89
- Additional Contributors:
90
- - Hang Wu (Applied Machine Learning, ByteDance)
91
- - Xia Xiao (Seed Foundation Code, ByteDance)
92
- - Yifan Sun (Seed Foundation Code, ByteDance)
93
- - Shen Zheng (Seed Foundation Code, ByteDance)
94
- - Kai Shen (Seed Foundation Code, ByteDance)
 
1
+ # BFS-Prover Tactic Generator
2
 
3
+ This repository contains the latest tactic generator model checkpoint from BFS-Prover, a state-of-the-art theorem proving system. While the full BFS-Prover system integrates multiple components for scalable theorem proving, we are releasing the core tactic generation model that achieved competitive performance on formal mathematics tasks.
4
 
5
  ## Model Details
6
 
7
+ - Base Model: Qwen2.5-Math-7B
8
+ - Training Approach:
9
+ - Supervised Fine-Tuning (SFT) on state-tactic pairs
10
+ - Direct Preference Optimization (DPO) using compiler feedback
11
+ - Training Data Sources:
12
+ - Mathlib (via LeanDojo)
13
+ - Lean-Github repositories
14
+ - Lean-Workbook
15
+ - Autoformalized NuminaMath-CoT dataset
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  ## Performance
18
 
19
+ When integrated into the full BFS-Prover system, this tactic generator model achieved
20
+ 72.54% success rate on MiniF2F test set accumulatively.
 
 
 
 
21
 
22
  ## Usage
23
 
24
  ```python
25
+ # Example code for loading and using the tactic generator model
26
  from transformers import AutoModelForCausalLM, AutoTokenizer
27
 
28
+ model = AutoModelForCausalLM.from_pretrained("path/to/bfsprover-tactic-generator")
29
+ tokenizer = AutoTokenizer.from_pretrained("path/to/bfsprover-tactic-generator")
 
 
 
 
 
 
 
 
 
 
30
 
31
+ # Input format example:
32
+ prompt = f"{Lean4 TacticState}" + ":::"
33
+ inputs = tokenizer(prompt, return_tensors="pt")
34
+ outputs = model.generate(**inputs)
35
+ tactic = tokenizer.decode(outputs[0])
36
+ ```
 
37
 
38
+ ## System Requirements
39
 
40
+ - Compatible with Hugging Face Transformers library
41
+ - Recommended: 16GB+ GPU memory for inference
 
42
 
43
  ## Citation
44
 
45
+ If you use this model in your research, please cite our paper:
46
+
47
  ```bibtex
48
+ @article{xin2025bfs,
49
  title={BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Proving},
50
  author={Xin, Ran and Xi, Chenguang and Yang, Jie and Chen, Feng and Wu, Hang and Xiao, Xia and Sun, Yifan and Zheng, Shen and Shen, Kai},
51
+ journal={arXiv preprint arXiv:2502.03438},
52
+ year={2025}
53
  }
54
  ```
55
 
56
+ ## License
57
+
58
+ https://choosealicense.com/licenses/apache-2.0/
59
 
60
+ ## Contact
 
 
 
 
61
 
62
+ For questions and feedback about the tactic generator model, please contact:
63
+ - Ran Xin ([email protected])
64
+ - Kai Shen ([email protected])