elkaioptimization/vllm-nvfp4-cuda-13
elkaioptimization
vLLM NVFP4 CUDA 13 - Blackwell-optimized vLLM for NVFP4/FP8 serving
下载次数: 0状态：社区镜像维护者：elkaioptimization仓库类型：镜像最近更新：6 个月前
让 AI 帮你使用轩辕镜像？ · 展开查看说明 · 点击收起说明
如果你使用 DeepSeek、元宝 AI、Copilot、ChatGPT 等 AI 工具，协助配置轩辕镜像、编写 docker pull 命令、修改 Docker Compose 镜像地址、配置镜像加速、排查镜像拉取失败、分析报错日志等问题，请先让 AI 阅读并遵守轩辕镜像的规则文档。
只需在 AI 对话中先发送下面这句话即可：
请先完整阅读并严格遵守以下文档中的全部规则与要求：
https://xuanyuan.cloud/agents.md
在未充分阅读并理解该文档前，不要生成任何命令、配置、修改建议、故障排查方案或技术回答。后续所有输出都必须严格以该文档中的规范为最高优先级执行。查看 agents.md 用法指南与完整示范。国内用户首推 元宝 AI、DeepSeek 的深度思考模式，不推荐豆包 AI；Cursor 等编辑器可在对话 @ 该链接，或加入 User Rules。 若 AI 无法访问外链，可 打开说明文档 复制全文粘贴。文档会随站点更新，复制内容可能过期，建议定期检查。
🦌 ELK-AI | vLLM NVFP4 Base
Base Image | NVFP4 Ready | Blackwell Native
https://img.shields.io/badge/NVFP4_Ready-Quantized-FF6F00?style=flat-square&logo=nvidia](https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13) https://img.shields.io/badge/-_Parameters-LLM-blue?style=flat-square&logo=pytorch](https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13) https://img.shields.io/badge/Varies_VRAM-GPU-green?style=flat-square&logo=nvidia](https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13) https://img.shields.io/badge/v***-red?style=flat-square&logo=v](https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13) https://img.shields.io/badge/CUDA-13.0-76B900?style=flat-square&logo=nvidia](https://developer.nvidia.com/cuda-toolkit) https://img.shields.io/badge/Blackwell-SM121-7B2D8E?style=flat-square&logo=nvidia](https://www.nvidia.com/dgx-spark) https://img.shields.io/badge/H100-Optimized-76B900?style=flat-square&logo=nvidia](https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13) https://img.shields.io/badge/A100-Optimized-76B900?style=flat-square&logo=nvidia](https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13)
Mutaz Al Awamleh | ELK-AI
🧠 About
Foundation image with pre-compiled FlashInfer, Mamba SSM, and NVFP4 support. Build your own optimized models.
🚀 Why This Image?
We rebuilt the entire inference stack from scratch with CUDA 13.0.
| Problem | Our Solution |
|---|---|
| FlashInfer compilation 2+ hrs | Pre-compiled for SM80-SM121 |
| Mamba SSM CUDA mismatches | Pre-built for all architectures |
| 50+ undocumented env vars | Battle-tested configuration |
| Days of CUDA graph tuning | Optimized out of the box |
Result: From WEEKS of setup to 30 SECONDS.
💻 Hardware Requirements
| Requirement | Minimum |
|---|---|
| VRAM | Varies |
| GPU | Any NVIDIA GPU |
| CUDA | 12.0+ |
📦 Specs
| Spec | Value |
|---|---|
| Parameters | - |
| Quantization | NVFP4 Ready |
| Size | 11.8 GB (was 15.0 GB) |
| Context | Model-dependent |
| Speed | Optimized |
🏗️ 7-Layer Optimization Stack
+-------------------------------------------------------------+ | ELK-AI GROUND-UP OPTIMIZATION STACK | +-------------------------------------------------------------+ | Layer 7: NVFP4 | 21% memory reduction | | Layer 6: FP8 KV-Cache | 2x context length | | Layer 5: FlashInfer 0.2.6 | 3x faster decoding | | Layer 4: CUDA Graphs | 6x faster warmup | | Layer 3: Mamba SSM 2.2.4 | State-space models | | Layer 2: vLLM V1 Engine | Optimal batching | | Layer 1: CUDA 13.0 + SM121 | Native Blackwell | +-------------------------------------------------------------+ | Base: NGC PyTorch 25.11 | cuBLAS 12.9 | TensorRT 10.x | +-------------------------------------------------------------+
🚀 Quick Start
bashdocker run --gpus all -p 8000:8000 elkaioptimization/vllm-nvfp4-cuda-13:2.5.0
🦌 More ELK-AI Models
| Model | Size | Link |
|---|---|---|
| Qwen3-VL-2B | 2.1 GB | https://hub.docker.com/r/elkaioptimization/qwen3-vl-2b-thinking-nvfp4-vllm-cuda13 |
| Qwen3-VL-4B | 4.2 GB | https://hub.docker.com/r/elkaioptimization/qwen3-vl-4b-thinking-nvfp4-vllm-cuda13 |
| Qwen3-VL-8B | 8.4 GB | https://hub.docker.com/r/elkaioptimization/qwen3-vl-8b-thinking-nvfp4-vllm-cuda13 |
| Nemotron3-30B | 31.5 GB | https://hub.docker.com/r/elkaioptimization/nemotron3-30b-nvfp4-vllm-cuda13 |
| Nemotron-VL-12B | 12.6 GB | https://hub.docker.com/r/elkaioptimization/nemotron-vl-12b-nvfp4-vllm-cuda13 |
| Devstral-24B | 53.8 GB | https://hub.docker.com/r/elkaioptimization/devstral-small-2-24b-fp8-vllm-cuda13 |
| vLLM Base | 11.8 GB | https://hub.docker.com/r/elkaioptimization/vllm-nvfp4-cuda-13 |
| vLLM Blackwell | 11.8 GB | https://hub.docker.com/r/elkaioptimization/vllm-blackwell-cuda13-optimized |
🦌 ELK-AI — From weeks of setup to 30 seconds
LinkedIn | Website | https://hub.docker.com/u/elkaioptimization
镜像拉取方式
您可以使用以下命令拉取该镜像。请将 <标签> 替换为具体的标签版本。如需查看所有可用标签版本，请访问 标签列表页面。
轩辕镜像加速拉取命令点我查看更多 vllm-nvfp4-cuda-13 镜像标签
docker pull docker.xuanyuan.run/elkaioptimization/vllm-nvfp4-cuda-13:<标签>
DockerHub 原生拉取命令
docker pull elkaioptimization/vllm-nvfp4-cuda-13:<标签>
镜像拉取常见问题
功能
错误码
用户好评
来自真实用户的反馈，见证轩辕镜像的优质服务
oldzhang
运维工程师
Linux服务器
5
"Docker访问体验非常流畅，大镜像也能快速完成下载。"