
如果你使用 DeepSeek、元宝 AI、Copilot、ChatGPT 等 AI 工具,协助配置轩辕镜像、编写 docker pull 命令、修改 Docker Compose 镜像地址、配置镜像加速、排查镜像拉取失败、分析报错日志等问题,请先让 AI 阅读并遵守轩辕镜像的规则文档。
只需在 AI 对话中先发送下面这句话即可:
请先完整阅读并严格遵守以下文档中的全部规则与要求:
https://xuanyuan.cloud/agents.md
在未充分阅读并理解该文档前,不要生成任何命令、配置、修改建议、故障排查方案或技术回答。后续所有输出都必须严格以该文档中的规范为最高优先级执行。查看 agents.md 用法指南与完整示范。国内用户首推 元宝 AI、DeepSeek 的深度思考模式,不推荐豆包 AI;Cursor 等编辑器可在对话 @ 该链接,或加入 User Rules。 若 AI 无法访问外链,可 打开说明文档 复制全文粘贴。文档会随站点更新,复制内容可能过期,建议定期检查。
Maintained by:
https://spark.apache.org/committers.html
Where to get help:
https://spark.apache.org/community.html
Dockerfile linkshttps://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java21-python3-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java21-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java21-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java21-python3-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java17-python3-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java17-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java17-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/593280202af9dd8ca71ae8afa39698800d5670bc/4.1.2/scala2.13-java17-python3-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java21-python3-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java21-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java21-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java21-python3-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java17-python3-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java17-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java17-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/1e6fe4d2c19bd4ac1bfd3b1ec7c1f90ccd25cf12/4.0.2/scala2.13-java17-python3-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java17-python3-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java17-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java17-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java17-python3-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java11-python3-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java11-r-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java11-ubuntu/Dockerfile
https://github.com/apache/spark-docker/blob/dd49d350dc0ee517299bd79f4be369f1e39fb9c2/3.5.8/scala2.12-java11-python3-r-ubuntu/Dockerfile
Where to file issues:
https://issues.apache.org/jira/browse/SPARK
Supported architectures: (https://github.com/docker-library/official-images#architectures-other-than-amd64)
https://hub.docker.com/r/amd64/spark/, https://hub.docker.com/r/arm64v8/spark/
Published image artifact details:
https://github.com/docker-library/repo-info/blob/master/repos/spark (https://github.com/docker-library/repo-info/commits/master/repos/spark)
(image metadata, transfer size, etc)
Image updates:
https://github.com/docker-library/official-images/issues?q=label%3Alibrary%2Fspark
https://github.com/docker-library/official-images/blob/master/library/spark (https://github.com/docker-library/official-images/commits/master/library/spark)
Source of this description:
https://github.com/docker-library/docs/tree/master/spark (https://github.com/docker-library/docs/commits/master/spark)
Apache Spark™ is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, pandas API on Spark for pandas workloads, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing.
!https://raw.githubusercontent.com/docker-library/docs/a16cd1ae80c04193c029a686d3006c95edb81594/spark/logo.png
You can find the latest Spark documentation, including a programming guide, on the https://spark.apache.org/documentation.html. This README file only contains basic setup instructions.
The easiest way to start using Spark is through the Scala shell:
consoledocker run -it spark /opt/spark/bin/spark-shell
Try the following command, which should return 1,000,000,000:
scalascala> spark.range(1000 * 1000 * 1000).count()
The easiest way to start using PySpark is through the Python shell:
consoledocker run -it spark:python3 /opt/spark/bin/pyspark
And run the following command, which should also return 1,000,000,000:
python>>> spark.range(1000 * 1000 * 1000).count()
The easiest way to start using R on Spark is through the R shell:
consoledocker run -it spark:r /opt/spark/bin/sparkR
https://spark.apache.org/docs/latest/running-on-kubernetes.html
See more in https://github.com/apache/spark-docker/blob/master/OVERVIEW.md#environment-variable
Apache Spark, Spark, Apache, the Apache feather logo, and the Apache Spark project logo are trademarks of The Apache Software Foundation.
Licensed under the https://www.apache.org/licenses/LICENSE-2.0.
As with all Docker images, these likely also contain other software which may be under other licenses (such as Bash, etc from the base distribution, along with any direct or indirect dependencies of the primary software being contained).
Some additional license information which was able to be auto-detected might be found in https://github.com/docker-library/repo-info/tree/master/repos/spark.
As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.
您可以使用以下命令拉取该镜像。请将 <标签> 替换为具体的标签版本。如需查看所有可用标签版本,请访问 标签列表页面。
来自真实用户的反馈,见证轩辕镜像的优质服务
以下是 spark 相关的常用 Docker 镜像,适用于 不同场景 等不同场景: