Update README.md
Browse files
README.md
CHANGED
@@ -18,7 +18,8 @@ tags:
|
|
18 |
|
19 |
# BFS-Prover Tactic Generator
|
20 |
|
21 |
-
This repository contains the latest tactic generator model checkpoint from BFS-Prover, a state-of-the-art theorem proving system. While the full BFS-Prover system integrates multiple components for scalable theorem proving, we are releasing the core tactic generation model that achieved
|
|
|
22 |
|
23 |
## Model Details
|
24 |
|
@@ -39,19 +40,16 @@ BFS-Prover achieves state-of-the-art performance on the MiniF2F test benchmark.
|
|
39 |
|
40 |
| Prover System | Search Method | Critic Model | Tactic Budget | Score |
|
41 |
|---------------|---------------|--------------|---------------|--------|
|
42 |
-
| BFS-Prover
|
43 |
-
| BFS-Prover
|
44 |
| HunyuanProver | BFS | Yes | 600×8×400 | 68.4% |
|
45 |
| InternLM2.5-StepProver | BFS | Yes | 256×32×600 | 65.9% |
|
46 |
| DeepSeek-Prover-V1.5* | MCTS | No | 32×16×400 | 63.5% |
|
47 |
|
48 |
-
*Note: DeepSeek-Prover-V1.5 uses whole-proof generation method; tactic budget decomposed for comparison.
|
49 |
|
50 |
### Key Advantages
|
51 |
-
- Achieves better performance without requiring a critic model
|
52 |
-
-
|
53 |
-
- Shows strong scaling with increased search passes
|
54 |
-
- Benefits from DPO training using compiler feedback
|
55 |
|
56 |
## Usage
|
57 |
|
@@ -59,20 +57,22 @@ BFS-Prover achieves state-of-the-art performance on the MiniF2F test benchmark.
|
|
59 |
# Example code for loading and using the tactic generator model
|
60 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
61 |
|
62 |
-
model = AutoModelForCausalLM.from_pretrained("
|
63 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
64 |
|
65 |
-
# Input format example:
|
66 |
-
|
|
|
|
|
67 |
inputs = tokenizer(prompt, return_tensors="pt")
|
68 |
outputs = model.generate(**inputs)
|
69 |
-
tactic = tokenizer.decode(outputs[0])
|
70 |
-
```
|
71 |
|
72 |
-
|
73 |
-
|
74 |
-
|
75 |
-
|
|
|
76 |
|
77 |
## Citation
|
78 |
|
|
|
18 |
|
19 |
# BFS-Prover Tactic Generator
|
20 |
|
21 |
+
This repository contains the latest tactic generator model checkpoint from BFS-Prover, a state-of-the-art theorem proving system. While the full BFS-Prover system integrates multiple components for scalable theorem proving, we are releasing the core tactic generation model that achieved state-of-the-art performance on formal mathematics tasks. Given a tactic state in Lean4, the model generates a tactic that transforms the current proof state into a new state, progressively working towards completing the proof.
|
22 |
+
|
23 |
|
24 |
## Model Details
|
25 |
|
|
|
40 |
|
41 |
| Prover System | Search Method | Critic Model | Tactic Budget | Score |
|
42 |
|---------------|---------------|--------------|---------------|--------|
|
43 |
+
| BFS-Prover | BFS | No | Accumulative | **72.95%** |
|
44 |
+
| BFS-Prover | BFS | No | 2048×2×600 | **70.83% ± 0.89%** |
|
45 |
| HunyuanProver | BFS | Yes | 600×8×400 | 68.4% |
|
46 |
| InternLM2.5-StepProver | BFS | Yes | 256×32×600 | 65.9% |
|
47 |
| DeepSeek-Prover-V1.5* | MCTS | No | 32×16×400 | 63.5% |
|
48 |
|
|
|
49 |
|
50 |
### Key Advantages
|
51 |
+
- Achieves better performance without requiring a critic model (value function)
|
52 |
+
- Combined with simpler search method (BFS) rather than MCTS
|
|
|
|
|
53 |
|
54 |
## Usage
|
55 |
|
|
|
57 |
# Example code for loading and using the tactic generator model
|
58 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
59 |
|
60 |
+
model = AutoModelForCausalLM.from_pretrained("bytedance-research/BFS-Prover")
|
61 |
+
tokenizer = AutoTokenizer.from_pretrained("bytedance-research/BFS-Prover")
|
62 |
|
63 |
+
# Input format example:
|
64 |
+
state = "h : x = y + 2 ⊢ x - 1 = y + 1"
|
65 |
+
sep = ":::"
|
66 |
+
prompt = state + sep
|
67 |
inputs = tokenizer(prompt, return_tensors="pt")
|
68 |
outputs = model.generate(**inputs)
|
69 |
+
tactic = tokenizer.decode(outputs[0], skip_special_tokens=True).split(sep)[1]
|
|
|
70 |
|
71 |
+
# Example input-output:
|
72 |
+
# Input state: "h : x = y + 2 ⊢ x - 1 = y + 1"
|
73 |
+
# Model output: "{input state}:::simp [h]"
|
74 |
+
# Final tactic: "simp [h]"
|
75 |
+
```
|
76 |
|
77 |
## Citation
|
78 |
|