RanXinByteDance commited on
Commit
cb6b8cc
·
verified ·
1 Parent(s): b7d5988

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -18
README.md CHANGED
@@ -18,7 +18,8 @@ tags:
18
 
19
  # BFS-Prover Tactic Generator
20
 
21
- This repository contains the latest tactic generator model checkpoint from BFS-Prover, a state-of-the-art theorem proving system. While the full BFS-Prover system integrates multiple components for scalable theorem proving, we are releasing the core tactic generation model that achieved competitive performance on formal mathematics tasks.
 
22
 
23
  ## Model Details
24
 
@@ -39,19 +40,16 @@ BFS-Prover achieves state-of-the-art performance on the MiniF2F test benchmark.
39
 
40
  | Prover System | Search Method | Critic Model | Tactic Budget | Score |
41
  |---------------|---------------|--------------|---------------|--------|
42
- | BFS-Prover (Accumulative) | BFS | No | - | **72.95%** |
43
- | BFS-Prover (This Work) | BFS | No | 2048×2×600 | **70.83% ± 0.89%** |
44
  | HunyuanProver | BFS | Yes | 600×8×400 | 68.4% |
45
  | InternLM2.5-StepProver | BFS | Yes | 256×32×600 | 65.9% |
46
  | DeepSeek-Prover-V1.5* | MCTS | No | 32×16×400 | 63.5% |
47
 
48
- *Note: DeepSeek-Prover-V1.5 uses whole-proof generation method; tactic budget decomposed for comparison.
49
 
50
  ### Key Advantages
51
- - Achieves better performance without requiring a critic model
52
- - Uses simpler search method (BFS) compared to MCTS
53
- - Shows strong scaling with increased search passes
54
- - Benefits from DPO training using compiler feedback
55
 
56
  ## Usage
57
 
@@ -59,20 +57,22 @@ BFS-Prover achieves state-of-the-art performance on the MiniF2F test benchmark.
59
  # Example code for loading and using the tactic generator model
60
  from transformers import AutoModelForCausalLM, AutoTokenizer
61
 
62
- model = AutoModelForCausalLM.from_pretrained("path/to/bfsprover-tactic-generator")
63
- tokenizer = AutoTokenizer.from_pretrained("path/to/bfsprover-tactic-generator")
64
 
65
- # Input format example:
66
- prompt = f"{Lean4 TacticState}" + ":::"
 
 
67
  inputs = tokenizer(prompt, return_tensors="pt")
68
  outputs = model.generate(**inputs)
69
- tactic = tokenizer.decode(outputs[0])
70
- ```
71
 
72
- ## System Requirements
73
-
74
- - Compatible with Hugging Face Transformers library
75
- - Recommended: 16GB+ GPU memory for inference
 
76
 
77
  ## Citation
78
 
 
18
 
19
  # BFS-Prover Tactic Generator
20
 
21
+ This repository contains the latest tactic generator model checkpoint from BFS-Prover, a state-of-the-art theorem proving system. While the full BFS-Prover system integrates multiple components for scalable theorem proving, we are releasing the core tactic generation model that achieved state-of-the-art performance on formal mathematics tasks. Given a tactic state in Lean4, the model generates a tactic that transforms the current proof state into a new state, progressively working towards completing the proof.
22
+
23
 
24
  ## Model Details
25
 
 
40
 
41
  | Prover System | Search Method | Critic Model | Tactic Budget | Score |
42
  |---------------|---------------|--------------|---------------|--------|
43
+ | BFS-Prover | BFS | No | Accumulative | **72.95%** |
44
+ | BFS-Prover | BFS | No | 2048×2×600 | **70.83% ± 0.89%** |
45
  | HunyuanProver | BFS | Yes | 600×8×400 | 68.4% |
46
  | InternLM2.5-StepProver | BFS | Yes | 256×32×600 | 65.9% |
47
  | DeepSeek-Prover-V1.5* | MCTS | No | 32×16×400 | 63.5% |
48
 
 
49
 
50
  ### Key Advantages
51
+ - Achieves better performance without requiring a critic model (value function)
52
+ - Combined with simpler search method (BFS) rather than MCTS
 
 
53
 
54
  ## Usage
55
 
 
57
  # Example code for loading and using the tactic generator model
58
  from transformers import AutoModelForCausalLM, AutoTokenizer
59
 
60
+ model = AutoModelForCausalLM.from_pretrained("bytedance-research/BFS-Prover")
61
+ tokenizer = AutoTokenizer.from_pretrained("bytedance-research/BFS-Prover")
62
 
63
+ # Input format example:
64
+ state = "h : x = y + 2 ⊢ x - 1 = y + 1"
65
+ sep = ":::"
66
+ prompt = state + sep
67
  inputs = tokenizer(prompt, return_tensors="pt")
68
  outputs = model.generate(**inputs)
69
+ tactic = tokenizer.decode(outputs[0], skip_special_tokens=True).split(sep)[1]
 
70
 
71
+ # Example input-output:
72
+ # Input state: "h : x = y + 2 ⊢ x - 1 = y + 1"
73
+ # Model output: "{input state}:::simp [h]"
74
+ # Final tactic: "simp [h]"
75
+ ```
76
 
77
  ## Citation
78