README.md

Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


note - bnb 8bits
- Model creator: https://huggingface.co/jinee/
- Original model: https://huggingface.co/jinee/note/




Original model description:
---
license: cc-by-nc-sa-4.0
language:
- en
tags:
- medical
---


# NOTE: Notable generation Of patient Text summaries through Efficient approach based on direct preference optimization

The discharge summary (DS) is a crucial document in the patient journey, as it encompasses all events from multiple visits, medications, varied imaging/laboratory tests, surgery/procedures, and admissions/discharge. 
Providing a summary of the patient’s progress is crucial, as it significantly influences future care and planning. 
Consequently, clinicians face the laborious and resource-intensive task of manually collecting, organizing, and combining all the necessary data for a DS. 
Therefore, we propose NOTE, which stands for “Notable generation Of patient Text summaries through an Efficient approach based on direct preference optimization (DPO)”. 
NOTE is based on MIMIC-III and summarizes a single hospitalization of a patient. Patient events are sequentially combined and used to generate a DS for each hospitalization. 
To demonstrate the practical application of the developed NOTE, we provide a web page-based demonstration software. In the future, we will aim to deploy the software available for actual use by clinicians in hospital.
NOTE can be utilized to generate various summaries not only discharge summaries but also throughout a patient's journey, thereby alleviating the labor-intensive workload of clinicians and aiming for increased efficiency.


## Model Description

- **Model type:** MistralForCausalLM
- **Language(s) (NLP):** English
- **License:** [CC-BY-NC-SA](https://creativecommons.org/licenses/by-nc-sa/4.0/)
- **Finetuned from model:** [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)

## Model Sources

- **Paper:** [NOTE](https://arxiv.org/abs/2402.11882)
- **Demo:** [NOTE-DEMO](https://huggingface.co/spaces/jinee/note-demo)

## Usage

~~~python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model = AutoModelForCausalLM.from_pretrained("jinee/note", load_in_4bit=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("jinee/note")
tokenizer.padding_side = 'right'
tokenizer.add_eos_token = True
tokenizer.pad_token = tokenizer.eos_token
tokenizer.add_eos_token, tokenizer.add_bos_token

instruction = '''
As a doctor, you need to create a discharge summary based on input data. 
Never change the dates or numbers in the input data and use them as is. And please follow the format below for your report. 
Also, never make up information that is not in the input data, and write a report only with information that can be identified from the input data.

1. Patient information (SUBJECT_ID, HADM_ID, hospitalization and discharge date, hospitalization period, gender, date of birth, age, allergy)
2. Diagnostic information and past history (if applicable)
3. Surgery or procedure information
4. Significant medication administration during hospitalization and discharge medication history
5. Meaningful lab tests during hospitalization
6. Summary of significant text records/notes
7. Discharge outcomes and treatment plan
8. Overall summary of at least 500 characters in lines including the above contents
'''
torch.cuda.empty_cache()

def generation(model, tokenizer, input_data):
    pipe = pipeline('text-generation', 
                   model = model,
                   tokenizer = tokenizer,
                   torch_dtype=torch.bfloat16,
                   device_map = 'auto')
    global instruction
       
    sequences = pipe(
        f"[INST]{instruction}: {input_data} [/INST]",
        do_sample=True,
        max_new_tokens=1024, 
        temperature=0.7, 
        top_k=50, 
        top_p=0.95,
        early_stopping =True,
        num_return_sequences=1,)

    text = sequences[0]['generated_text']
    start_index = text.find('[/INST]')
    if start_index != -1:
        summary_ = text[start_index + len('[/INST]'):]
        return(summary_)
    else:
        return("'[summary_] 'is not founded.")

~~~


## Dataset

The model has been trained on a [MIMIC-III](https://physionet.org/content/mimiciii/1.4/), a comprehensive and freely accssible de-identified medical database.
Access to this databased requires a number of steps to obtain permission. 


## Training and Hyper-parameters

### List of LoRA config 
based on [Parameter-Efficient Fine-Tuning (PEFT)](https://github.com/huggingface/peft)

Parameter | SFT | DPO 
:------:| :------:| :------:
r | 16 | 16 
lora alpha | 16 | 16 
lora dropout | 0.05 | 0.05  
target | q, k, v, o, gate | q, k, v, o, gate 


### List of Training arguments
based on [Transformer Reinforcement Learning (TRL)](https://github.com/huggingface/trl)

Parameter | SFT | DPO 
:------:| :------:| :------:
early stopping patience | 3 | 3 
early stopping threshold | 0.0005 | 0.0005 
train epochs | 20 | 3 
per device train batch size | 4 | 1 
per device eval batch size | 8 (default) | 1 
optimizer | paged adamw 8bit | paged adamw 8bit
lr scheduler | cosine | cosine 
wramup ratio | 0.3 | 0.1
gradient accumulation step | 2 | 2 
evaluation strategy | step | step 
eval step | 10 | 5 


### Experimental setup
- **Ubuntu 20.04 LTS**
- **2 NVIDIA GeForce RTX 3090 GPUs**
- **Python**: 3.8.10
- **Pytorch**:2.0.1+cu118
- **Transformer**:4.35.2


## Limitations

The model was limited in character count for comparison with the existing T5 model, but it is planned to be expanded in future research.
Additionally, further research on prompting engineering is needed due to it producing different results with the same instructions.
Most metrics for evaluating summarization and generation tasks were somewhat challenging to apply to our study, and while we attempted to address this through the ChatGPT4 Assistant API, future research will be based on feedback from clinicians.

## Non-commercial use
These models are available exclusively for research purposes and are not intended for commercial use.

<!-- ## Citation

**BibTeX:**
-->


## INMED DATA
INMED DATA is developing large language models (LLMs) specifically tailored for medical applications. For more information, please visit our website [TBD].