dustynv/ollama
dustynv
GitHub仓库dusty-nv/jetson-containers中的Ollama LLM包，是为NVIDIA Jetson嵌入式平台设计的容器化解决方案，旨在简化大型语言模型（LLM）的部署与运行流程，支持多种主流LLM模型，充分利用Jetson设备的硬件加速能力，适用于边缘AI计算、智能终端开发等场景，为开发者提供便捷高效的本地化LLM部署工具。
9 次收藏下载次数: 0状态：社区镜像维护者：dustynv仓库类型：镜像最近更新：1 年前
让 AI 帮你使用轩辕镜像？ · 展开查看说明 · 点击收起说明
如果你使用 DeepSeek、元宝 AI、Copilot、ChatGPT 等 AI 工具，协助配置轩辕镜像、编写 docker pull 命令、修改 Docker Compose 镜像地址、配置镜像加速、排查镜像拉取失败、分析报错日志等问题，请先让 AI 阅读并遵守轩辕镜像的规则文档。
只需在 AI 对话中先发送下面这句话即可：
请先完整阅读并严格遵守以下文档中的全部规则与要求：
https://xuanyuan.cloud/agents.md
在未充分阅读并理解该文档前，不要生成任何命令、配置、修改建议、故障排查方案或技术回答。后续所有输出都必须严格以该文档中的规范为最高优先级执行。查看 agents.md 用法指南与完整示范。国内用户首推 元宝 AI、DeepSeek 的深度思考模式，不推荐豆包 AI；Cursor 等编辑器可在对话 @ 该链接，或加入 User Rules。 若 AI 无法访问外链，可 打开说明文档 复制全文粘贴。文档会随站点更新，复制内容可能过期，建议定期检查。
ollama
CONTAINERSIMAGESRUNBUILD
- Ollama from https://github.com/ollama/ollama with CUDA enabled (found under
/bin/ollama) - Thanks to https://github.com/remy415 for getting Ollama working on Jetson and contributing the Dockerfile (https://github.com/dusty-nv/jetson-containers/pull/465)
Ollama Server
First, start the local Ollama server as a daemon in the background, either of these ways:
# models cached under jetson-containers/data jetson-containers run --name ollama $(autotag ollama) # models cached under your user's home directory docker run --runtime nvidia -it -rm --network=host -v ~/ollama:/ollama -e OLLAMA_MODELS=/ollama dustynv/ollama:r36.2.0
You can then run the ollama client in the same container (or a different one if desired). The default docker run CMD of the ollama container is /start_ollama, which starts the ollama server in the background and returns control to the user. The ollama server logs are saved under your mounted jetson-containers/data/logs directory for monitoring them outside the containers.
Setting the $OLLAMA_MODELS environment variable as shown above will change where ollama downloads the models to. By default, this is under your jetson-containers/data/models/ollama directory which is automatically mounted by jetson-containers run.
Ollama Client
Start the Ollama CLI front-end with your desired model (for example: mistral 7b)
# if running inside the same container as launched above /bin/ollama run mistral # if launching a new container for the client in another terminal jetson-containers run $(autotag ollama) /bin/ollama run mistral
Or you can run the client outside container by installing Ollama's binaries for arm64 (without CUDA, which only the server needs)
# download the latest ollama release for arm64 into /bin sudo wget https://github.com/ollama/ollama/releases/download/$(git ls-remote --refs --sort="version:refname" --tags https://github.com/ollama/ollama | cut -d/ -f3- | sed 's/-rc.*//g' | tail -n1)/ollama-linux-arm64 -O /bin/ollama sudo chmod +x /bin/ollama # use the client like normal (outside container) /bin/ollama run mistral
Open WebUI
To run https://github.com/open-webui/open-webui server for client browsers to connect to, use the open-webui container:
docker run -it --rm --network=host --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main
You can then navigate your browser to http://JETSON_IP:8080, and create a fake account to login (these credentials are only stored locally)
Memory Usage
| Model | Quantization | Memory (MB) |
|---|---|---|
| https://huggingface.co/TheBloke/Llama-2-7B-GGUF | llama-2-7b.Q4_K_S.gguf | 5,268 |
| https://huggingface.co/TheBloke/Llama-2-13B-GGUF | llama-2-13b.Q4_K_S.gguf | 8,609 |
| https://huggingface.co/TheBloke/LLaMA-30b-GGUF | llama-30b.Q4_K_S.gguf | 19,045 |
| https://huggingface.co/TheBloke/Llama-2-70B-GGUF | llama-2-70b.Q4_K_S.gguf | 37,655 |
CONTAINERS
ollama | |
|---|---|
| Requires | L4T ['>=34.1.0'] |
| Dependencies | https://github.com/dusty-nv/jetson-containers/tree/master/packages/build/build-essential https://github.com/dusty-nv/jetson-containers/tree/master/packages/cuda/cuda |
| Dependants | https://github.com/dusty-nv/jetson-containers/tree/master/packages/rag/llama-index |
| Dockerfile | https://github.com/dusty-nv/jetson-containers/tree/master/packages/llm/ollama/Dockerfile |
| Images | https://hub.docker.com/r/dustynv/ollama/tags (2024-04-25, 5.4GB)https://hub.docker.com/r/dustynv/ollama/tags (2024-04-25, 3.9GB) |
CONTAINER IMAGES
| Repository/Tag | Date | Arch | Size |
|---|---|---|---|
| https://hub.docker.com/r/dustynv/ollama/tags | 2024-04-25 | arm64 | 5.4GB |
| https://hub.docker.com/r/dustynv/ollama/tags | 2024-04-25 | arm64 | 3.9GB |
Container images are compatible with other minor versions of JetPack/L4T:
• L4T R32.7 containers can run on other versions of L4T R32.7 (JetPack 4.6+)
• L4T R35.x containers can run on other versions of L4T R35.x (JetPack 5.1+)
RUN CONTAINER
To start the container, you can use https://github.com/dusty-nv/jetson-containers/tree/master/docs/run.md and https://github.com/dusty-nv/jetson-containers/tree/master/docs/run.md#autotag, or manually put together a https://docs.docker.com/engine/reference/commandline/run/ command:
bash# automatically pull or build a compatible container image jetson-containers run $(autotag ollama) # or explicitly specify one of the container images above jetson-containers run dustynv/ollama:r35.4.1 # or if using 'docker run' (specify image and mounts/ect) sudo docker run --runtime nvidia -it --rm --network=host dustynv/ollama:r35.4.1
https://github.com/dusty-nv/jetson-containers/tree/master/docs/run.md forwards arguments to https://docs.docker.com/engine/reference/commandline/run/ with some defaults added (like
--runtime nvidia, mounts a/datacache, and detects devices)
https://github.com/dusty-nv/jetson-containers/tree/master/docs/run.md#autotag finds a container image that's compatible with your version of JetPack/L4T - either locally, pulled from a registry, or by building it.
To mount your own directories into the container, use the https://docs.docker.com/engine/reference/commandline/run/#volume or https://docs.docker.com/engine/reference/commandline/run/#volume flags:
bashjetson-containers run -v /path/on/host:/path/in/container $(autotag ollama)
To launch the container running a command, as opposed to an interactive shell:
bashjetson-containers run $(autotag ollama) my_app --abc xyz
You can pass any options to it that you would to https://docs.docker.com/engine/reference/commandline/run/, and it'll print out the full command that it constructs before executing it.
BUILD CONTAINER
If you use https://github.com/dusty-nv/jetson-containers/tree/master/docs/run.md#autotag as shown above, it'll ask to build the container for you if needed. To manually build it, first do the https://github.com/dusty-nv/jetson-containers/tree/master/docs/setup.md, then run:
bashjetson-containers build ollama
The dependencies from above will be built into the container, and it'll be tested during. Run it with https://github.com/dusty-nv/jetson-containers/tree/master/jetson_containers/build.py for build options.
更多相关 Docker 镜像与资源
以下是 dustynv/ollama 相关的常用 Docker 镜像，适用于 不同场景 等不同场景：
- ollama/ollama Docker 镜像说明
- shinejh0528/ollama Docker 镜像说明（Ollama 社区变体（shinejh0528），适合本地大模型推理与 API）
- eisai/ollama Docker 镜像说明（Ollama 社区变体（eisai），适合本地大模型推理）
- mthreads/ollama Docker 镜像说明（Ollama 镜像变体，适合 Ollama、Vllm、Llama.cpp 相关部署场景）
- infiniflow/ragflow Docker 镜像说明（RAG 知识库系统）
镜像拉取方式
您可以使用以下命令拉取该镜像。请将 <标签> 替换为具体的标签版本。如需查看所有可用标签版本，请访问 标签列表页面。
DockerHub 原生拉取命令
docker pull dustynv/ollama:<标签>
镜像拉取常见问题
功能
错误码
用户好评
来自真实用户的反馈，见证轩辕镜像的优质服务
oldzhang
运维工程师
Linux服务器
5
"Docker访问体验非常流畅，大镜像也能快速完成下载。"