Installation

Package Installation

Install the package:

pip install ollama-classifier

Or with uv:

uv add ollama-classifier

httpx and pydantic are required dependencies and are installed automatically. The ollama Python SDK is optional — install it only if you use the Ollama backend:

pip install "ollama-classifier[ollama]"

The vLLM, SGLang, and llama.cpp backends communicate over HTTP using httpx (already a core dependency), so no extra install is needed for them.

Prerequisites

Before using ollama-classifier, you need at least one inference backend:

Ollama backend

Ollama ≥0.12 installed and running

Download and install Ollama from: https://ollama.com/download

Note

Ollama runtime v0.12 or later is required for logprobs support.

The Ollama Python SDK (optional dependency):

pip install "ollama-classifier[ollama]"

A model pulled

Pull a model to use for classification:
```
ollama pull llama3.2
```
You can use any model that supports JSON structured output. Recommended models:
- llama3.2 - Fast and capable
- llama3.1 - Larger and more capable
- mistral - Good balance of speed and quality
- qwen2.5 - Excellent for classification tasks

vLLM backend

Install vLLM: https://docs.vllm.ai/en/latest/getting_started/installation.html

Start a local server:

python -m vllm.entrypoints.openai.api_server \
    --model meta-llama/Llama-3.2-3B-Instruct \
    --host 0.0.0.0 --port 8000

SGLang backend

Install SGLang: https://docs.sglang.ai/start/install.html

Start a local server:

python -m sglang.launch_server \
    --model-path meta-llama/Llama-3.2-3B-Instruct \
    --host 0.0.0.0 --port 30000

llama.cpp backend

Download or build llama-server: https://github.com/ggerganov/llama.cpp

Start a local server:

./llama-server -m model.gguf --host 0.0.0.0 --port 8080 -c 4096

Development Installation

To contribute to the project or run the documentation locally:

git clone https://github.com/paluigi/ollama-classifier.git
cd ollama-classifier
uv sync --extra docs --extra ollama

Building Documentation Locally

After installing with the docs extra:

cd docs
sphinx-build -b html . _build/html

Then open _build/html/index.html in your browser.