WestlakeNLP
/

CycleResearcher-72B

@@ -1,332 +1,341 @@
----
-license: other
-license_name: cycleresearcher-license
-license_link: LICENSE
-base_model:
-- Qwen/Qwen2.5-72B-Instruct
-language:
-- en
-- zh
-- ja
-- ko
-- fr
-- de
-metrics:
-- accuracy
-extra_gated_prompt: "You agree to not use the model for generating research papers for direct submission or official publication without explicit disclosure of AI assistance."
-extra_gated_fields:
-  First Name: text
-  Last Name: text
-  Country: country
-  Affiliation: text
-  Academic Status:
-    type: select
-    options:
-      - Student
-      - Researcher
-      - Professor
-      - Industry Professional
-      - Other
-  Specific date: date_picker
-  I want to use this model for:
-    type: select
-    options:
-      - Research Ideation
-      - Literature Review Assistance
-      - Experimental Design Planning
-      - Methodology Development
-      - Draft Writing Practice
-      - Research Validation
-      - Hypothesis Generation
-      - Reference Organization
-      - Writing Improvement
-      - Academic Training
-      - Research Planning
-      - Supplementary Tool
-      - label: Other
-        value: other
-  Research Field:
-    type: select
-    options:
-      - Machine Learning
-      - Computer Vision
-      - Natural Language Processing
-      - Robotics
-      - Other AI Fields
-      - label: Other
-        value: other
-  geo: ip_location
-  I agree to NOT submit papers generated by this model without proper disclosure: checkbox
-  I agree to explicitly acknowledge AI assistance in any resulting publications: checkbox
-  I agree to use this model for research assistance purposes only: checkbox
-  I understand that experimental results generated by this model are simulated: checkbox
-extra_gated_button_content: Submit
-library_name: transformers
-datasets:
-- WestlakeNLP/Research-14K
----
-# CycleResearcher: Automated Research via Reinforcement Learning with Iterative Feedback
-HomePage: https://wengsyx.github.io/Researcher/
-This model is the safe version of CycleResearcher.
-## Model Specifications
-|       Model Name        |                 Pre-training Language Model                  |                           HF Link                            |
-| :---------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
-| CycleResearcher-ML-12B  | [Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) | [🤗 link](https://huggingface.co/WestlakeNLP/CycleResearcher-ML-12B) |
-| CycleResearcher-ML-72B  | [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) | [🤗 link](https://huggingface.co/WestlakeNLP/CycleResearcher-ML-72B) |
-| CycleResearcher-ML-123B | [Mistral-Large-2](https://huggingface.co/mistralai/Mistral-Large-Instruct-2407) | [🤗 link](https://huggingface.co/WestlakeNLP/CycleResearcher-ML-123B) |
-## Model Info
-The CycleResearcher model series includes two main variants:
-1. **ML Series**: Specifically trained for machine learning research, including computer vision (CV), natural language processing (NLP), and multimedia (MM)
-2. **Science Series**: Extended to broader scientific domains (Coming soon)
-All models have undergone extensive training on our Research-8k dataset and are optimized using the CycleReviewer feedback loop. According to our license, **all models and their derivatives cannot be used for generating papers without proper disclosure of AI assistance.** We also provide FastDetectGPT-based tools to detect potential misuse of these models.
-**Model Release Date**: October 2024
-**Knowledge Cutoff Date**: October 2024
-### Open Source License
-The code in this repository is open-sourced under the Apache-2.0 license. The model weights are open-sourced under the CycleResearcher-License.
-### Model Performance
-Results on research paper generation evaluated by CycleReviewer:
-| Paper Type | Source | Avg Min Score ↑ | Avg Max Score ↑ | Avg Score ↑ | Accept Rate |
-|------------|--------|----------------|-----------------|-------------|-------------|
-| Conference Accept Papers† | Human Expert | **3.91** | **6.98** | **5.69** | **100.00%** |
-| Preprint Papers | Human Expert | 3.24 | 6.62 | 5.24 | 29.63% |
-| AI Scientist | AI | 2.20 | 5.70 | 4.31 | 0.00% |
-| CycleResearcher-12B | AI | 3.47 | **6.75** | 5.36 | **35.13%** |
-| CycleResearcher-72B| AI | **3.65** | 6.58 | **5.38** | 33.64% |
-| CycleResearcher-123B | AI | 3.31 | 6.42 | 5.13 | 21.19% |
-### Detecting misuse of CycleResearcher
-To ensure the responsible use of our models, we implemented the Fast-DetectGPT method to classify whether a paper is machine-generated. Detection performance comparison across different formats. The human samples are from the test sets of Research-8k and Reviewer-5k.
-| Model | Format | Accuracy | F1 Score |
-|-------|---------|-----------|-----------|
-| Researcher-12B | Paper | 98.38% | 98.37 |
-| Researcher-72B | Paper | 97.52% | 97.49 |
-| Researcher-123B | Paper | 98.88% | 98.87 |
-## Installation
-```bash
-pip install cycleresearcher
-pip install torch>=2.0.0
-pip install transformers>=4.44.0
-pip install vllm  # Optional, for faster inference
-```
-## Requirements
-- Python >= 3.8
-- PyTorch >= 2.0.0
-- Transformers >= 4.44.0
-- CUDA >= 11.8 (for GPU acceleration)
-## System Requirements
-Recommended configurations for different model sizes:
-| Model | Recommended Config | Minimum Config |
-|-------|--------------------|----------------|
-| CycleResearcher-12B | 2x H100 80G        | 1x H100 80G    |
-| CycleResearcher-72B | 8x H100 80G        | 4x H100 80G    |
-| CycleResearcher-123B | 8x H100 80G        | 8x H100 80G    |
-## Quick Start
-### Using Transformers
-```python
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-# Initialize model
-model_name = "WestlakeNLP/CycleResearcher-12B"
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-    torch_dtype=torch.float16,
-    device_map="auto",
-    max_memory={i: "24GiB" for i in range(torch.cuda.device_count())},
-)
-# Generation parameters
-generation_config = {
-    "max_length": 19000,
-    "temperature": 0.1,
-    "top_p": 0.95,
-    "pad_token_id": None,
-    "do_sample": True,
-}
-# Prepare system prompt and input
-system_prompt = """You are a research assistant AI tasked with generating a scientific paper based on provided literature. Follow these steps:
-1. Analyze the given References.
-2. Identify gaps in existing research to establish the motivation for a new study.
-3. Propose a main idea for a new research work.
-4. Write the paper's main content in LaTeX format, including:
- - Title
- - Abstract
- - Introduction
- - Related Work
- - Methods/
-5. Generate experimental setup details in JSON format to guide researchers.
-6. After receiving experimental results in JSON format, analyze them.
-7. Complete the paper by writing:
- - Results
- - Discussion
- - Conclusion
- - Contributions
-Ensure all content is original, academically rigorous, and follows standard scientific writing conventions."""
-# Reference input should be in BibTeX format
-references = """@article{Qiu2020PretrainedMF,
-  title={Pre-trained models for natural language processing: A survey},
-  author={Xipeng Qiu and Tianxiang Sun and Yige Xu and Yunfan Shao and Ning Dai and Xuanjing Huang},
-  journal={Science China Technological Sciences},
-  year={2020},
-  volume={63},
-  pages={1872 - 1897}
-}
-@article{Long2022VisionandLanguagePM,
-  title={Vision-and-Language Pretrained Models: A Survey},
-  author={Siqu Long and Feiqi Cao and Soyeon Caren Han and Haiqing Yang},
-  journal={IJCAI},
-  year={2022},
-}
-@inproceedings{Klicpera2019DiffusionIG,
-  title={Diffusion Improves Graph Learning},
-  author={Johannes Klicpera and Stefan Wei{\ss}enberger and Stephan G{\"u}nnemann},
-  booktitle={Neural Information Processing Systems},
-  year={2019}
-The above content represents the relevant literature in this field. Please analyze it and provide the motivation and main idea. Then, provide the Title, Abstract, Introduction, Related Work, and Methods sections in LaTeX format.
-"""
-messages = [
-    {"role": "system", "content": system_prompt},
-    {"role": "user", "content": references}
-]
-# Generate paper
-prompt = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
-inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-outputs = model.generate(**inputs, **generation_config)
-generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
-```
-### Using VLLM (Recommended for faster inference)
-```python
-from vllm import LLM, SamplingParams
-# Initialize model with VLLM
-model = LLM(
-    model="WestlakeNLP/CycleResearcher-12B",
-    tensor_parallel_size=8,
-    max_model_len=15000,
-    gpu_memory_utilization=0.95,
-)
-# Generation parameters
-sampling_params = SamplingParams(
-    temperature=0.4,
-    top_p=0.95,
-    max_tokens=4096
-)
-# Generate paper
-outputs = model.generate([prompt], sampling_params)
-```
-## Input Data Format
-CycleResearcher expects reference input in BibTeX format with abstracts. Example format:
-```bibtex
-@article{example2023,
-    title = {Sample Paper Title},
-    author = {Author, A. and Author, B.},
-    journal = {Journal Name},
-    year = {2024},
-    abstract = {This is a sample abstract that provides context...}
-}
-Abstract: This is a sample abstract that provides context...
-@article{example2024,
-    title = {Sample Paper Title},
-    author = {Author, A. and Author, B.},
-    journal = {Journal Name},
-    year = {2024},
-}
-```
-## Output Format
-The model generates output with the following structure:
-```python
-{
-    'title': 'Paper title',
-    'abstract': 'Paper abstract',
-    'latex': 'Main paper content in LaTeX format',
-    'motivation': 'Research motivation',
-    'idea': 'Main research idea',
-    'Experimental_Setup': 'Experiment configuration (JSON/text)',
-    'Experimental_results': 'Results and findings (JSON/text)',
-    'generated_text': 'Complete raw generated text'
-}
-```
-## Training and Evaluation Datasets
-- Research-8k: Contains 12,696 training and 802 test samples
-- Review-5k: Contains 4,970 papers with over 16,000 reviewer comments
-To request access to these datasets, please contact [email protected].
-## License
-The code is released under the Apache 2.0 license. Use of the models is subject to the CycleResearcher-License agreement.
-## Citation
-```bibtex
-@inproceedings{cycleresearcher2024,
-  title={CycleResearcher: Improving Automated Research via Automated Review},
-  author={Anonymous Authors},
-  booktitle={International Conference on Learning Representations},
-  year={2025}
-}
-```
-## Contact
-For questions and feedback, please:
-- Open an issue on GitHub
-- Contact [email protected]
----
 **Note**: This is a research preview release. Features and capabilities may be updated frequently.

+---
+license: other
+license_name: cycleresearcher-license
+license_link: LICENSE
+base_model:
+- Qwen/Qwen2.5-72B-Instruct
+language:
+- zho
+- eng
+- fra
+- spa
+- por
+- deu
+- ita
+- rus
+- jpn
+- kor
+- vie
+- tha
+- ara
+metrics:
+- accuracy
+extra_gated_prompt: You agree to not use the model for generating research papers
+  for direct submission or official publication without explicit disclosure of AI
+  assistance.
+extra_gated_fields:
+  First Name: text
+  Last Name: text
+  Country: country
+  Affiliation: text
+  Academic Status:
+    type: select
+    options:
+    - Student
+    - Researcher
+    - Professor
+    - Industry Professional
+    - Other
+  Specific date: date_picker
+  I want to use this model for:
+    type: select
+    options:
+    - Research Ideation
+    - Literature Review Assistance
+    - Experimental Design Planning
+    - Methodology Development
+    - Draft Writing Practice
+    - Research Validation
+    - Hypothesis Generation
+    - Reference Organization
+    - Writing Improvement
+    - Academic Training
+    - Research Planning
+    - Supplementary Tool
+    - label: Other
+      value: other
+  Research Field:
+    type: select
+    options:
+    - Machine Learning
+    - Computer Vision
+    - Natural Language Processing
+    - Robotics
+    - Other AI Fields
+    - label: Other
+      value: other
+  geo: ip_location
+  I agree to NOT submit papers generated by this model without proper disclosure: checkbox
+  I agree to explicitly acknowledge AI assistance in any resulting publications: checkbox
+  I agree to use this model for research assistance purposes only: checkbox
+  I understand that experimental results generated by this model are simulated: checkbox
+extra_gated_button_content: Submit
+library_name: transformers
+datasets:
+- WestlakeNLP/Research-14K
+---
+# CycleResearcher: Automated Research via Reinforcement Learning with Iterative Feedback
+HomePage: https://wengsyx.github.io/Researcher/
+This model is the safe version of CycleResearcher.
+## Model Specifications
+|       Model Name        |                 Pre-training Language Model                  |                           HF Link                            |
+| :---------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| CycleResearcher-ML-12B  | [Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) | [🤗 link](https://huggingface.co/WestlakeNLP/CycleResearcher-ML-12B) |
+| CycleResearcher-ML-72B  | [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) | [🤗 link](https://huggingface.co/WestlakeNLP/CycleResearcher-ML-72B) |
+| CycleResearcher-ML-123B | [Mistral-Large-2](https://huggingface.co/mistralai/Mistral-Large-Instruct-2407) | [🤗 link](https://huggingface.co/WestlakeNLP/CycleResearcher-ML-123B) |
+## Model Info
+The CycleResearcher model series includes two main variants:
+1. **ML Series**: Specifically trained for machine learning research, including computer vision (CV), natural language processing (NLP), and multimedia (MM)
+2. **Science Series**: Extended to broader scientific domains (Coming soon)
+All models have undergone extensive training on our Research-8k dataset and are optimized using the CycleReviewer feedback loop. According to our license, **all models and their derivatives cannot be used for generating papers without proper disclosure of AI assistance.** We also provide FastDetectGPT-based tools to detect potential misuse of these models.
+**Model Release Date**: October 2024
+**Knowledge Cutoff Date**: October 2024
+### Open Source License
+The code in this repository is open-sourced under the Apache-2.0 license. The model weights are open-sourced under the CycleResearcher-License.
+### Model Performance
+Results on research paper generation evaluated by CycleReviewer:
+| Paper Type | Source | Avg Min Score ↑ | Avg Max Score ↑ | Avg Score ↑ | Accept Rate |
+|------------|--------|----------------|-----------------|-------------|-------------|
+| Conference Accept Papers† | Human Expert | **3.91** | **6.98** | **5.69** | **100.00%** |
+| Preprint Papers | Human Expert | 3.24 | 6.62 | 5.24 | 29.63% |
+| AI Scientist | AI | 2.20 | 5.70 | 4.31 | 0.00% |
+| CycleResearcher-12B | AI | 3.47 | **6.75** | 5.36 | **35.13%** |
+| CycleResearcher-72B| AI | **3.65** | 6.58 | **5.38** | 33.64% |
+| CycleResearcher-123B | AI | 3.31 | 6.42 | 5.13 | 21.19% |
+### Detecting misuse of CycleResearcher
+To ensure the responsible use of our models, we implemented the Fast-DetectGPT method to classify whether a paper is machine-generated. Detection performance comparison across different formats. The human samples are from the test sets of Research-8k and Reviewer-5k.
+| Model | Format | Accuracy | F1 Score |
+|-------|---------|-----------|-----------|
+| Researcher-12B | Paper | 98.38% | 98.37 |
+| Researcher-72B | Paper | 97.52% | 97.49 |
+| Researcher-123B | Paper | 98.88% | 98.87 |
+## Installation
+```bash
+pip install cycleresearcher
+pip install torch>=2.0.0
+pip install transformers>=4.44.0
+pip install vllm  # Optional, for faster inference
+```
+## Requirements
+- Python >= 3.8
+- PyTorch >= 2.0.0
+- Transformers >= 4.44.0
+- CUDA >= 11.8 (for GPU acceleration)
+## System Requirements
+Recommended configurations for different model sizes:
+| Model | Recommended Config | Minimum Config |
+|-------|--------------------|----------------|
+| CycleResearcher-12B | 2x H100 80G        | 1x H100 80G    |
+| CycleResearcher-72B | 8x H100 80G        | 4x H100 80G    |
+| CycleResearcher-123B | 8x H100 80G        | 8x H100 80G    |
+## Quick Start
+### Using Transformers
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+# Initialize model
+model_name = "WestlakeNLP/CycleResearcher-12B"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.float16,
+    device_map="auto",
+    max_memory={i: "24GiB" for i in range(torch.cuda.device_count())},
+)
+# Generation parameters
+generation_config = {
+    "max_length": 19000,
+    "temperature": 0.1,
+    "top_p": 0.95,
+    "pad_token_id": None,
+    "do_sample": True,
+}
+# Prepare system prompt and input
+system_prompt = """You are a research assistant AI tasked with generating a scientific paper based on provided literature. Follow these steps:
+1. Analyze the given References.
+2. Identify gaps in existing research to establish the motivation for a new study.
+3. Propose a main idea for a new research work.
+4. Write the paper's main content in LaTeX format, including:
+ - Title
+ - Abstract
+ - Introduction
+ - Related Work
+ - Methods/
+5. Generate experimental setup details in JSON format to guide researchers.
+6. After receiving experimental results in JSON format, analyze them.
+7. Complete the paper by writing:
+ - Results
+ - Discussion
+ - Conclusion
+ - Contributions
+Ensure all content is original, academically rigorous, and follows standard scientific writing conventions."""
+# Reference input should be in BibTeX format
+references = """@article{Qiu2020PretrainedMF,
+  title={Pre-trained models for natural language processing: A survey},
+  author={Xipeng Qiu and Tianxiang Sun and Yige Xu and Yunfan Shao and Ning Dai and Xuanjing Huang},
+  journal={Science China Technological Sciences},
+  year={2020},
+  volume={63},
+  pages={1872 - 1897}
+}
+@article{Long2022VisionandLanguagePM,
+  title={Vision-and-Language Pretrained Models: A Survey},
+  author={Siqu Long and Feiqi Cao and Soyeon Caren Han and Haiqing Yang},
+  journal={IJCAI},
+  year={2022},
+}
+@inproceedings{Klicpera2019DiffusionIG,
+  title={Diffusion Improves Graph Learning},
+  author={Johannes Klicpera and Stefan Wei{\ss}enberger and Stephan G{\"u}nnemann},
+  booktitle={Neural Information Processing Systems},
+  year={2019}
+The above content represents the relevant literature in this field. Please analyze it and provide the motivation and main idea. Then, provide the Title, Abstract, Introduction, Related Work, and Methods sections in LaTeX format.
+"""
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": references}
+]
+# Generate paper
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, **generation_config)
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+```
+### Using VLLM (Recommended for faster inference)
+```python
+from vllm import LLM, SamplingParams
+# Initialize model with VLLM
+model = LLM(
+    model="WestlakeNLP/CycleResearcher-12B",
+    tensor_parallel_size=8,
+    max_model_len=15000,
+    gpu_memory_utilization=0.95,
+)
+# Generation parameters
+sampling_params = SamplingParams(
+    temperature=0.4,
+    top_p=0.95,
+    max_tokens=4096
+)
+# Generate paper
+outputs = model.generate([prompt], sampling_params)
+```
+## Input Data Format
+CycleResearcher expects reference input in BibTeX format with abstracts. Example format:
+```bibtex
+@article{example2023,
+    title = {Sample Paper Title},
+    author = {Author, A. and Author, B.},
+    journal = {Journal Name},
+    year = {2024},
+    abstract = {This is a sample abstract that provides context...}
+}
+Abstract: This is a sample abstract that provides context...
+@article{example2024,
+    title = {Sample Paper Title},
+    author = {Author, A. and Author, B.},
+    journal = {Journal Name},
+    year = {2024},
+}
+```
+## Output Format
+The model generates output with the following structure:
+```python
+{
+    'title': 'Paper title',
+    'abstract': 'Paper abstract',
+    'latex': 'Main paper content in LaTeX format',
+    'motivation': 'Research motivation',
+    'idea': 'Main research idea',
+    'Experimental_Setup': 'Experiment configuration (JSON/text)',
+    'Experimental_results': 'Results and findings (JSON/text)',
+    'generated_text': 'Complete raw generated text'
+}
+```
+## Training and Evaluation Datasets
+- Research-8k: Contains 12,696 training and 802 test samples
+- Review-5k: Contains 4,970 papers with over 16,000 reviewer comments
+To request access to these datasets, please contact [email protected].
+## License
+The code is released under the Apache 2.0 license. Use of the models is subject to the CycleResearcher-License agreement.
+## Citation
+```bibtex
+@inproceedings{cycleresearcher2024,
+  title={CycleResearcher: Improving Automated Research via Automated Review},
+  author={Anonymous Authors},
+  booktitle={International Conference on Learning Representations},
+  year={2025}
+}
+```
+## Contact
+For questions and feedback, please:
+- Open an issue on GitHub
+- Contact [email protected]
+---
 **Note**: This is a research preview release. Features and capabilities may be updated frequently.