Python chat

Beta

Target SDK design

python

from botconnector import Client

client = Client()
model = client.llm("<publisher>/<model>")
response = model.chat("Explain GGUF in one paragraph")
print(response.text)

The source SDK is implemented and live-tested, but the PyPI package is not published. The example assumes the source boundary is on your PYTHONPATH.

Working today via OpenAI-compatible API

python

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:11435/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="<publisher>/<model>",
    messages=[
        {"role": "system", "content": "You are a concise technical assistant."},
        {"role": "user", "content": "Explain GGUF in one paragraph"},
    ],
)
print(resp.choices[0].message.content)

For the current terminal workflow, use BCCLI.

Notes

The model field must match the loaded model id; list what is loaded with botconnector ps --json.
Full endpoint reference: POST /v1/chat/completions.