BotConnector
Sign in Workspace Open App

Tool use

Tool Use: beta · public · verified

What is it?

Tool (function) calling lets a model emit a structured request — name plus JSON arguments — instead of a plain reply, so your code can execute a real function and feed the result back. BotConnector exposes this through the OpenAI-compatible tools field.

When should I use it?

  • Letting assistants fetch live data (weather, prices, databases) or perform actions.
  • Building agents that need more than text generation.

Prerequisites

  • A model tagged for tool/function calling — filter the catalog by Tools.
  • A chat template that renders tool-call syntax (tool-aware GGUF templates).
  • Runtime support for tool parsing. Experimental

Minimal working example

bash
curl http://127.0.0.1:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<publisher>/<model>",
    "messages": [{"role":"user","content":"What is the weather in Jakarta?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

A tool-capable model answers with finish_reason: "tool_calls" and a structured call your code executes, then you continue the conversation with the tool result as a role: "tool" message.

Configuration/options

  • Tool definitions follow the OpenAI function schema.
  • Whether the model decides to call a tool depends on prompt and model quality.

Limitations

  • Model/template dependent — a model without tool training will hallucinate call syntax or ignore tools. The badge is inferred; verify with your model. See Capabilities.
  • Not all chat templates in the wild render tool calls; check the model card.
  • Parallel tool calls depend on template support.

Troubleshooting

  • No tool calls emitted → model likely lacks tool training; try a Tools-tagged model.
  • Malformed JSON in tool calls → lower temperature, or larger quantization.
  • Details: Model problems.