mindthemath/nomic-embed-v1.5

source code: https://github.com/mindthemath/nomic-api-rs

nomic-serve

A fast Rust server for generating text and image embeddings using https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 and https://huggingface.co/nomic-ai/nomic-embed-vision-v1.5 models via ONNX Runtime.

Quick Start

Pick either the cpu or gpu tag (adding --gpus all for the latter), and start the server with:

bash
docker run --rm -p 8080:8080 mindthemath/nomic-embed-v1.5:cpu

# Test
curl -X POST localhost:8080/embed \
  -H 'content-type: application/json' \
  -d '{"input": "Hello world"}'

There are also :..-quantizeed variants of the images which ship with smaller model artifacts at the cost of reduced precision.

API

Interactive documentation available at /docs (Swagger UI).

GET /health

Returns health status and model availability.

Response:

json
{
  "status": "OK",
  "text_model": true,
  "vision_model": true
}

GET /info

Returns server information including model paths and configuration.

Response:

json
{
  "averaging": "geometric",
  "load_cuda": false,
  "txt_model": "models/txt/model_quantized.onnx",
  "tokenizer": "models/txt/tokenizer.json",
  "img_model": "models/img/model_quantized.onnx"
}

POST /embed (or /txt/embed)

Generate embedding for a single text.

Request:

json
{"input": "Hello world", "dim": 768}

• input (required): Text to embed
• dim (optional): Embedding dimension (1-768). Defaults to 768. Supports https://huggingface.co/blog/matryoshka - use smaller dimensions for faster similarity search.

Response:

json
{
  "embedding": [0.123, 0.456, ...],
  "tokens": 4,
  "time_ms": 12.34
}

Example with reduced dimension:

json
{"input": "Hello world", "dim": 128}

Returns a 128-dimensional embedding (faster similarity search, slightly lower quality).

POST /batch (or /txt/batch)

Generate embeddings for multiple texts.

Request:

json
{"inputs": ["Hello world", "Goodbye world"], "dim": 8}

• inputs (required): List of texts to embed
• dim (optional): Embedding dimension (1-768). Defaults to 768. Applied to all embeddings in the batch.

Response:

json
{
  "embeddings": [[0.123, 0.456, ...], [0.789, -0.123, ...]],
  "tokens": [4, 5],
  "time_ms": 45.67
}

POST /img/embed

Generate embedding for a single image.

Request:

json
{
  "content": "https://example.com/image.jpg",
  "dim": 768
}

• content (required): Image as URL, data URL (data:image/jpeg;base64,...), or raw base64
• dim (optional): Embedding dimension (1-768). Defaults to 768.

Response:

json
{
  "embedding": [0.123, 0.456, ...],
  "time_ms": 89.12
}

Limits: Maximum 20MB per image (compressed). Supports JPEG, PNG, GIF, WebP, BMP, TIFF.

POST /img/batch

Generate embeddings for multiple images.

Request:

json
{
  "contents": [
    "https://example.com/cat.jpg",
    "data:image/png;base64,iVBORw0KGgo..."
  ],
  "dim": 768
}

Response:

json
{
  "embeddings": [[0.123, ...], [0.789, ...]],
  "time_ms": 178.45
}

POST /img/stats

Extract image statistics including EXIF metadata and color analysis.

Request:

json
{
  "content": "https://example.com/image.jpg",
  "averaging_method": "geometric"
}

• content (required): Image as URL, data URL (data:image/jpeg;base64,...), or raw base64
• averaging_method (optional): Color averaging method - arithmetic or geometric (defaults to AVERAGING env var or geometric)

Response:

json
{
  "exif_data": {
    "Make": "Canon",
    "Model": "EOS 5D",
    "DateTime": "2024:01:01 12:00:00"
  },
  "color_data": {
    "avg_color": {
      "rgb": [0.5, 0.3, 0.2],
      "hex": "#804d33",
      "method": "geometric"
    },
    "dominant_color": {
      "rgb": [0.6, 0.4, 0.3],
      "hex": "#99664d"
    }
  },
  "time_ms": 12.34
}

Limits: Maximum 20MB per image (compressed). Supports JPEG, PNG, GIF, WebP, BMP, TIFF.

GET /docs

Swagger UI documentation page.

GET /openapi.json

OpenAPI 3.1.0 schema.

Configuration

CORS Configuration

By default, the server allows requests from localhost only (for local development):

• http://localhost:3000 / http://localhost:8080
• http://127.0.0.1:3000 / http://127.0.0.1:8080

Production deployment - set allowed origins explicitly:

bash
# Allow specific origins (comma-separated, no wildcards)
CORS_ORIGINS="https://example.com,https://app.example.com,https://api.example.com" ./nomic-serve

# Docker
docker run -p 8080:8080 \
  -e CORS_ORIGINS="https://example.com,https://app.example.com" \
  mindthemath/nomic-text-v1.5-rs:latest-cpu

Security notes:

• Origins must be explicitly listed (no wildcard support)
• If CORS_ORIGINS is set but contains invalid values, falls back to localhost defaults (never permissive)
• Invalid origins are silently ignored and logged

Disable CORS entirely (allows all origins - use only for internal APIs):

bash
DISABLE_CORS=1 ./nomic-serve

To modify the default allowed origins, edit DEFAULT_CORS_ORIGINS in src/main.rs.

---

Matryoshka Embeddings

The nomic-embed-text-v1.5 model supports Matryoshka embeddings - variable-dimension embeddings that maintain quality at reduced dimensions. Use the dim parameter to truncate embeddings for faster similarity search.

Benefits:

• Faster similarity search: Smaller vectors = faster distance calculations
• Reduced storage: Store fewer dimensions per embedding
• Quality preserved: Lower dimensions maintain high quality for most use cases

Recommended dimensions:

• 768 (default): Full quality, best for fine-grained tasks
• 512: ~99% quality, good balance
• 256: ~95% quality, faster search
• 128: ~90% quality, very fast search
• 64: ~85% quality, fastest search

Example:

bash
# Full dimension (default)
curl -X POST localhost:8080/embed \
  -H 'content-type: application/json' \
  -d '{"input": "Hello world"}'

# Reduced dimension for faster search
curl -X POST localhost:8080/embed \
  -H 'content-type: application/json' \
  -d '{"input": "Hello world", "dim": 128}'

Important: All embeddings in a batch use the same dim value. For consistent similarity search, always use the same dim for all embeddings you compare.

Model Architecture

According to the https://static.nomic.ai/reports/2024_Nomic_Embed_Text_Technical_Report.pdf:

• Base architecture: BERT-based encoder with 137M parameters
• Context length: 8,192 tokens (extended from standard 512)
• Training: Multi-stage contrastive learning with 235M text pairs
• Features: Matryoshka Representation Learning for variable-dimension embeddings

Note: The model architecture itself supports batching (proven by FP32 and PyTorch implementations). The interference is specific to ONNX quantized models.

Why This Matters

For most embedding use cases (similarity search, clustering, RAG), embeddings need to be deterministic — the same text should always produce the same embedding. Cross-sample interference violates this:

embed("hello") alone     → [0.123, 0.456, ...]
embed("hello") + "world" → [0.089, 0.512, ...]  # Different!

This could cause:

• Inconsistent search results depending on what else was in the batch
• Non-reproducible experiments
• Subtle bugs that are hard to diagnose

Sequential Processing is Correct (for Quantized Model)

By processing each text individually (batch_size=1) with the quantized model, we guarantee:

• Deterministic results: Same text → same embedding, always
• No cross-sample interference: Each text processed in isolation
• Correctness over speed: Throughput is lower, but results are reliable

Alternative: Use FP32 Model for Batching

If you need batching for text embeddings:

• Use FP32 model (model.onnx instead of model_quantized.onnx)
• FP32 batches perfectly (0.000000 difference, cosine similarity = 1.0)
• Trade-off: Larger model size (~375MB vs ~131MB) but enables batching

---

Dependencies

CPU inference: Standard C libraries (glibc, libstdc++). No GPU drivers or CUDA needed.

GPU inference: NVIDIA CUDA drivers and CUDA toolkit. The ***ary is built with CUDA support enabled by default (can be disabled by removing "cuda" feature from Cargo.toml).

Docker

Note: The --dns flags are recommended for image embedding endpoints (/img/embed, /img/batch) to ensure fast DNS resolution. Cloudflare DNS (1.1.1.1) is used for privacy and performance. Without DNS configuration, image URL fetching may be slow (10+ seconds) due to Docker's default DNS configuration.

Image Size (as shown by docker images):

• CPU image: ~358MB
  • ***ary: 38MB
  • Text model files (model_quantized.onnx + tokenizer.json): 132MB
  • Vision model (model_quantized.onnx): 93MB
  • Base image (debian:bookworm-slim): 74.8MB
  • Runtime dependencies (ca-certificates, libssl3, dumb-init): 9.2MB
  • Layer compression overhead: ~12MB
• GPU image: ~2.7GB
  • ***ary: 38MB
  • Text model files (model_quantized.onnx + tokenizer.json): 132MB
  • Vision model (model_quantized.onnx): 93MB
  • ONNX Runtime CUDA providers libraries: 196MB
  • CUDA runtime base image (nvidia/cuda:12.1.0-runtime-ubuntu22.04): 2.23GB
  • Runtime dependencies: 8MB
  • Layer compression overhead: ~100MB
  • Note: The -runtime variant is required (not -base) as it includes CUDA runtime libraries needed by ONNX Runtime's CUDA execution provider. The ONNX Runtime CUDA providers add significant size but are required for GPU inference.

Note: The CPU Docker image includes both text and vision models for full multimodal support. The GPU image is significantly larger due to the CUDA runtime base image (~2.23GB) required for GPU inference.

Model Info

Text Model:

• Model: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
• Embedding dimension: 768
• Max sequence length: 8,192 tokens
• Pooling: Mean pooling over non-padding tokens

Vision Model:

• Model: https://huggingface.co/nomic-ai/nomic-embed-vision-v1.5
• Embedding dimension: 768
• Input size: 224×224 (auto-resized)
• Pooling: CLS token extraction

Both models share the same embedding space via contrastive training, enabling direct comparison of text and image embeddings.

License: Apache 2.0

References

• https://static.nomic.ai/reports/2024_Nomic_Embed_Text_Technical_Report.pdf
• https://www.nomic.ai/blog/posts/nomic-embed-text-v1
• https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
• https://huggingface.co/nomic-ai/nomic-embed-vision-v1.5

License

MIT