empirischtech

/

Llama-3.1-8B-Instruct-MedQA

like 1

Follow

Empirisch Tech GmbH 6

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "empirischtech/Llama-3.1-8B-Instruct-MedQA"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"
model.to('cuda')

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What are the symptoms of diabetes?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(prompt)
eot_token_id = tokenizer.convert_tokens_to_ids("<|eot_id|>")
im_end_token_id = tokenizer.convert_tokens_to_ids("<|im_end|>")
# Tokenize the prompt
inputs = tokenizer(prompt, return_tensors="pt").to('cuda')

# Generate a response
with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs.input_ids,
        attention_mask=inputs.attention_mask,
        max_length=512,
        num_return_sequences=1,
        do_sample=True,
        temperature=1.2,
        top_p=0.95,
        repetition_penalty=1.1,
        eos_token_id=[eot_token_id, im_end_token_id, tokenizer.eos_token_id]  # List of token IDs to stop at
    )
# Decode the generated tokens to text
generated_text = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(generated_text)

Output:
context: A patient presents with fatigue and weakness. The history is not taken because it was cut short by her physician, who prescribed glucose as he suspected she may have an insulin deficiency. For how many days/week will you counsel this patient to check their fasting urine for glucose? When do you expect blood tests to check: hemoglobin A1c; insulin; and fasting plasma glucose? What medication would be appropriate if the results are as anticipated? Based on the above information, which of the following is a risk factor for acquiring diabetes mellitus type 2? CrossAxisAlignment
The answer is: Insulin - C7; Hemoglobin A1C - O2; Urine glucose strip test - Daily x 3 days during the third week of each month; Fasting Plasma Glucose - Monthly. Based on the question, which of the following is true of diabetes:groupon

What is going on here - or are some of my params really off?