intel/intel-optimized-text-embedding

Intel Optimized Container for Embeddings

The Intel Optimized Container for Embeddings is a lightweight text embeddig model that can be used for a variety of NLP tasks. The model is distilled from UAE-Large-v1 using the the sentence-transformers and Intel® Extension for Pytorch (IPEX) frameworks. It is a 23M parameter model with an input sequence length of 512 and output embedding size of 512. It achieves average accuracies of 46% and 82% on the MTEB Retrieval and STS tasks respectively. The model files and container source code can be found https://github.com/intel/Intel-Optimized-Container-for-Embeddings.

Run

This model is optimized for Intel® Xeon® Archicture using Intel® Extension for Pytorch (IPEX) and enables the use of the latest Intel® Advanced Matrix Extensions (AMX) for accelerated BF16 inference.

Run with built-in torchserve config:

bash
docker run --network=host --cap-add SYS_NICE -t -d --rm -p 7080:7080 --name=local_model intel-text-embedding:latest

Run with custom config:

bash
docker run --network=host --cap-add SYS_NICE -t -d --rm -p 7080:7080 -v ./config.properties:/home/ubuntu/config.properties --name=local_model intel-text-embedding:latest

Local Test

bash
curl -s -X POST \
  -H "Content-Type: application/json" \
  -d @./instances.json \
  http://localhost:7080/predictions/intel_embedding_model/

Training Datasets