kweg/pole-api

Pole assistant

Is a tool meant for interactions with your offline LLM assitants. The name stands for "field" in Polish but using the English meaning works for me as well!

"pole" means field in Polish.

This is early beta, made with ExDD methodology.

This early stage beta version is built upon the ExDD methodology - short for Excitement Driven Development, a method that keeps the thrill alive while coding. It's exciting, but also means we need to be extra vigilant for any hiccups or missing bits. While I did my best to make sure that all success paths work properly I cannot promise something won't break. Also some interactions are incomplite (there are no toast messages yet!). Every day I either add new feature or improve existing one

Running

Docker

For now the easiest way to try it out is to use docker compose. Images are on dockerhub.

services:
  pole-api:
    image: kweg/pole-api:0.1.3
    restart: unless-stopped
    volumes:
      - /path/to/db:/usr/src/app/db
    environment:
      CHROMA_SERVER: http://chroma_server
      OLLAMA_SERVER: http://olama_server
    ports:
      - 6550:3000
  pole-chat:
    restart: unless-stopped
    image: kweg/pole-chat:0.1.1
    environment:
      NODE_ENV: production
      REACT_APP_API: http://localhost:6550
    ports:
      - 6551:800

Variables

• CHROMA_SERVER - it's a path to your chroma instance. You can set it along as another service - for now Pole does not support credentials for chroma. This is security risk if you're exposing your server in any way. Have a look at chromadb documentation, there's also a dockerfile https://github.com/chroma-core/chroma/tree/main
• OLLAMA_SERVER - it's a path to your ollama instance. You can set it along as another service - for now this is only supported inference API. Choose your way to run it - https://github.com/ollama/ollama
• REACT_APP_API - it has to be address of the API service in most basic scenario it's exactly as on the sample

Volumes

• usr/src/app/db - is a file store, everything is stored as JSON files so you can have an easy access. If you choose not to persist this directory you'll loose all your data.

Setup

Setup is rather strightforward, you should run npm i and once it's done npm run build this should create build directory which contains .env file.

services:
  searxng:
      image: docker.io/searxng/searxng:latest
      restart: unless-stopped
      ports:
        - 8080:8080
      volumes:
        - /opt/searchxng:/etc/searxng:rw
      environment:
        - SEARXNG_BASE_URL=https://${SEARXNG_HOSTNAME:-localhost}/
      cap_drop:
        - ALL
      cap_add:
        - CHOWN
        - SETGID
        - SETUID
      logging:
        driver: "json-file"
        options:
          max-size: "1m"
          max-file: "1"
  chroma:
    image: ghcr.io/chroma-core/chroma:latest
    restart: unless-stopped
    environment:
      - IS_PERSISTENT=TRUE
    volumes:
      # Default configuration for persist_directory in chromadb/config.py
      # Currently it's located in "/chroma/chroma/"
      - /opt/chroma:/chroma/chroma/
    ports:
      - 8101:8000
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:8101/api/v1/heartbeat" ]
      interval: 30s
      timeout: 10s
      retries: 3

  pole-api:
    image: kweg/pole-api:0.4.6
    restart: unless-stopped
    volumes:
      - /data/db:/usr/src/app/db
      - /data/uploads:/uploads
    environment:
      NODE_ENV: production
      CHROMA_SERVER: http://localhost:8101
      OLLAMA_SERVER: http://localhost:11434
      OPENSEARCH_SERVER: http://localhost:8080
    ports:
      - 6550:3000 # REST API
      - 6552:3001 # Sockets API

  pole-chat:
    restart: unless-stopped
    image: kweg/pole-chat:0.4.6
    environment:
      NODE_ENV: production
      REACT_APP_API: http://localhost:6550
      REACT_APP_SOCKET: http://localhost:6552
    ports:
      - 6551:80