BotConnector
ID
warning
Legacy managed-runtime reference. Command examples below describe the older desktop/managed llama.cpp preview and are legacy, not the current BCCLI v0.4 command surface. Use BCCLI and Local AI for current instructions.

Chat

General conversational and text generation. This is the default category: instruct-tuned models that accept a message list and produce assistant replies.

When should I use a Chat model

  • Conversational assistants, Q&A over prompts, drafting and summarization.
  • Any workflow where a user or application sends text and expects text back.

Prerequisites

  • A chat/instruct GGUF downloaded — filter the catalog by Chat.
  • Runtime ready: botconnector runtime status — Runtime.

Minimal working example

powershell
botconnector get <publisher>/<model>@Q4_K_M
botconnector run <publisher>/<model>
botconnector chat

Or through the API:

bash
curl http://127.0.0.1:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"<publisher>/<model>","messages":[{"role":"user","content":"Hello"}]}'

Options

Limitations

  • Base (non-instruct) models complete text rather than answering; prefer instruct variants for chat.
  • Long conversations can fall out of the context window — earlier turns are lost.
  • Quality depends on quantization: Quantization.

Troubleshooting

Cut-off or repetitive replies, template mismatch symptoms: Model problems.