BotConnector
ID
warning
Legacy managed-runtime reference. Command examples below describe the older desktop/managed llama.cpp preview and are legacy, not the current BCCLI v0.4 command surface. Use BCCLI and Local AI for current instructions.

GPU problems

CUDA

SymptomFix
"CUDA error" at initUpdate NVIDIA driver; verify with nvidia-smi
VRAM OOMSmaller quant, lower context, or partial offload (Context and memory)
Slow despite CUDACheck other apps using VRAM; verify offload level with botconnector ps
Old GPU unsupportedTry Vulkan backend, else CPU

Vulkan

SymptomFix
"vulkan device not found"Update GPU driver; verify with vulkaninfo
Crash at initDriver coverage varies; try CPU to isolate
iGPU OOM (shared memory)Windows caps shared GPU memory (~half of RAM); use smaller quants

General GPU debugging

  1. Confirm the backend is actually active: botconnector runtime status.
  2. Fall back: botconnector runtime use cpu — if CPU works, the issue is GPU-specific.
  3. Update drivers before reporting; include nvidia-smi or vulkaninfo output.
  4. Try a tiny model (1B) on the GPU backend to separate "backend broken" from "model too big".

Offload notes

  • Full offload needs model + KV in VRAM.
  • Partial offload splits layers; PCIe transfer can make it slower than expected — benchmark both.

See also CUDA and Vulkan.