Qwen2.5-32B Instruct (GGUF)

Official adapter · maintained by ModelForgeLab

Qwen2.5-32B fine-tuned for long-form report writing and analysis, quantized to q5_k_m for a balance of quality and memory footprint. Requires ~22 GB RAM.

Usage

Load with Ollama:

# Download the adapter file, then create a Modelfile:
FROM qwen2.5-32b-instruct
ADAPTER ./qwen25-32b-instruct-gguf.gguf

# Build and run:
ollama create my-model -f Modelfile
ollama run my-model "Your prompt here"

Or with llama.cpp:

./llama-cli \
  -m base-model.gguf \
  --lora qwen25-32b-instruct-gguf.gguf \
  -p "Your prompt here"

Compatibility

• llama.cpp
• Ollama
• LM Studio