Llama 3 8B Python Typing

Published by @typed_python_ai · Community adapter

Adds PEP 484/526/604 type annotations to untyped Python code. Infers types from usage context, generates Protocol classes, and handles generic types and TypeVar correctly.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
base = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
model = PeftModel.from_pretrained(base, "modelforgelab/llama3-8b-python-typing-lora")

inputs = tokenizer("Your prompt here", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Compatibility

• transformers>=4.40
• vLLM