Embeddings

What is it?

Embedding models convert text into numerical vectors. Similar texts produce similar vectors, enabling semantic search, clustering and retrieval (RAG). They are not chat models — they take text in and return numbers.

When should I use one?

Local semantic search over documents, notes or code.
Building RAG pipelines without sending data anywhere.

Prerequisites

An embedding GGUF (feature-extraction pipeline) — filter the catalog by Embeddings.
A loaded embedding model in the runtime.

Minimal working example

bash

curl http://127.0.0.1:11435/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"<publisher>/<embedding-model>","input":["BotConnector runs local models","Local inference keeps data on device"]}'

Returns one vector per input string. Similar inputs yield high cosine similarity.

Configuration/options

Vectors have fixed dimensionality per model (e.g. 768, 1024) — mixing models means re-embedding your corpus.
Batch multiple inputs in one request for throughput.

Limitations

Not a chat model: do not point chat clients at embedding models.
Quantized embedding models lose some accuracy; embedding models are usually small enough to run at higher precision.
One loaded model at a time in preview — switching between chat and embedding models unloads the other. See Run a model.

Troubleshooting

Vector dimension errors, 404 for the model: API problems and Model problems.