datastax/astra-assistants

Astra Assistant API Service

A drop-in compatible service for the latest OpenAI Assistants API v2 (with streaming) with support for persistent threads, files, vector_stores, assistants, retreival, function calling and more using AstraDB (DataStax's db as a service offering powered by https://cassandra.apache.org/_/index.html and https://github.com/jbellis/jvector).

Supports dozens of third party LLM providers (or even local models) for both completion and embeddings (powered by https://github.com/BerriAI/litellm).

You can use our hosted Astra Assistants service, or host the open source API server yourself.

Run with Docker

with docker, first pull the image from docker hub

docker pull datastax/astra-assistants

or a specific version if you don't want latest:

docker pull datastax/astra-assistants:v0.2.18

then run (-p to map your docker port 8080 to your host port 8080):

docker run -p 8080:8080 datastax/astra-assistants

Docker-compose with ollama

For with docker-compose for integration with ollama clone the repo:

git clone ***:datastax/astra-assistants-api.git

and run the docker compose under examples:

cd examples/ollama/gpu # or examples/ollama/cpu for cpu only for gpu you need docker-toolkit

docker-compose up -d

you need to pull the model you want to ollama before using it

curl http://localhost:***/api/pull -d '{ "name": "deepseek-coder-v2" }'

your assistants client should route to the ollama container setting OLLAMA_API_BASE_URL. OLLAMA_API_BASE_URL should be set to [] if you are using docker-compose. If you are using ollama on your localhost you can set it to http://localhost: