Model problems
Output looks wrong
| Symptom | Likely cause | Fix |
|---|---|---|
| Model completes text instead of answering | Base (non-instruct) model | Use an instruct/chat variant (Chat models) |
| Repeated loops / gibberish | Quantization too small or repeat penalty | Try Q5/Q8 of the same model |
| Ignoring instructions in long chats | Context overflow — early turns dropped | Larger context or shorter history (Context and memory) |
| Thinking mixed into answers | Reasoning template behavior | Check the model card; template-specific parsing planned (Reasoning) |
Capability surprises
Badges are inferred — a model may be tagged Tools/Vision on the Hub but still fail locally when the runtime or template lacks support. The disclaimer: Capabilities.
- Tool calls never emitted → model/template lacks tool training: Tool use.
- Image input ignored → projector file missing: Vision.
- 404 calling an embedding model → load it first: Embeddings.
Template mismatches
Symptoms: strange tokens (<|im_start|>) visible in output, or answers starting mid-sentence. The model's chat template must match what the runtime applies. If the model card states a template the runtime does not support, that model may not work correctly in this build — check Model problems in runtime.
Model loaded but "wrong model" answers
Two models with the same name from different publishers exist. Verify the exact publisher/model id from botconnector ps --json.
Escalation
Include doctor --json, model id, quantization, backend and logs (Troubleshooting).