TypeScript streaming

Desain target SDK

typescript

import { BotConnectorClient } from "@botconnector/sdk";

const client = new BotConnectorClient();
const model = await client.llm("<publisher>/<model>");
for await (const chunk of model.stream("Jelaskan GGUF")) {
  process.stdout.write(chunk.text);
}

Source SDK sudah diimplementasikan dan diuji live, tetapi paket npm belum dipublikasikan.

Bekerja hari ini via API kompatibel OpenAI

Contoh ini memanggil BotConnector Core, runtime desktop, pada API lokalnya di 127.0.0.1:11435 — permukaan preview, bukan kontrak CLI publik. Untuk API hosted, gunakan https://api.botconnector.id/v1 seperti pada API kompatibel OpenAI.

typescript

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://127.0.0.1:11435/v1", apiKey: "not-needed" });
const stream = await client.chat.completions.create({
  model: "<publisher>/<model>",
  messages: [{ role: "user", content: "Jelaskan GGUF" }],
  stream: true,
});
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

Format wire: Streaming.