Chat

General conversational and text generation. This is the default category: instruct-tuned models that accept a message list and produce assistant replies.

When should I use a Chat model

• Conversational assistants, Q&A over prompts, drafting and summarization.
• Any workflow where a user or application sends text and expects text back.

Prerequisites

• A chat/instruct GGUF downloaded — filter the catalog by Chat.
• Runtime ready: botconnector runtime status — Runtime.

Minimal working example

botconnector get <publisher>/<model>@Q4_K_M
botconnector run <publisher>/<model>
botconnector chat

Or through the API:

curl http://127.0.0.1:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"<publisher>/<model>","messages":[{"role":"user","content":"Hello"}]}'

Options

• Context length at load time — Context and memory.
• System prompt via the system role — Chat.
• Streaming with "stream": true — Streaming.

Limitations

• Base (non-instruct) models complete text rather than answering; prefer instruct variants for chat.
• Long conversations roll out of context — earlier turns are lost.
• Quality depends on quantization: Quantization.

Troubleshooting

Cut-off or repetitive replies, template mismatch symptoms: Model problems.

Related

• Capabilities — badge meanings.
• Run a model.