radanalyticsio/radanalytics-r-spark
radanalyticsio
自动构建
OpenShift S2I builder image for R and Apache Spark using sparklyr with Oshinko management
下载次数: 0状态：自动构建维护者：radanalyticsio仓库类型：镜像最近更新：6 年前
让 AI 帮你使用轩辕镜像？ · 展开查看说明 · 点击收起说明
如果你使用 DeepSeek、元宝 AI、Copilot、ChatGPT 等 AI 工具，协助配置轩辕镜像、编写 docker pull 命令、修改 Docker Compose 镜像地址、配置镜像加速、排查镜像拉取失败、分析报错日志等问题，请先让 AI 阅读并遵守轩辕镜像的规则文档。
只需在 AI 对话中先发送下面这句话即可：
请先完整阅读并严格遵守以下文档中的全部规则与要求：
https://xuanyuan.cloud/agents.md
在未充分阅读并理解该文档前，不要生成任何命令、配置、修改建议、故障排查方案或技术回答。后续所有输出都必须严格以该文档中的规范为最高优先级执行。查看 agents.md 用法指南与完整示范。国内用户首推 元宝 AI、DeepSeek 的深度思考模式，不推荐豆包 AI；Cursor 等编辑器可在对话 @ 该链接，或加入 User Rules。 若 AI 无法访问外链，可 打开说明文档 复制全文粘贴。文档会随站点更新，复制内容可能过期，建议定期检查。
https://travis-ci.org/radanalyticsio/oshinko-s2i.svg?branch=master](https://travis-ci.org/radanalyticsio/oshinko-s2i) https://img.shields.io/docker/automated/radanalyticsio/radanalytics-pyspark.svg](https://hub.docker.com/r/radanalyticsio/radanalytics-pyspark) https://img.shields.io/docker/automated/radanalyticsio/radanalytics-pyspark-py36.svg](https://hub.docker.com/r/radanalyticsio/radanalytics-pyspark-py36) https://img.shields.io/docker/automated/radanalyticsio/radanalytics-java-spark.svg](https://hub.docker.com/r/radanalyticsio/radanalytics-java-spark) https://img.shields.io/docker/automated/radanalyticsio/radanalytics-scala-spark.svg](https://hub.docker.com/r/radanalyticsio/radanalytics-scala-spark)
oshinko-s2i
This is a place to put s2i images and utilities for Apache Spark application builders for OpenShift.
Complete versus incomplete images
There are two types of images that can be built from this repository, complete and incomplete
Complete images include a pre-selected Apache Spark distribution that is installed when the image is built.
Incomplete images contain radanalytics.io tooling but do not include a Spark distribution. With these images, users can perform s2i builds and produce images with Spark distributions of their choosing. See below for information on how to use the incomplete images.
Building the s2i images
The easiest way to build the s2i images is to use the makefiles provided:
# To build all images $ make # To build images individually $ make -f Makefile.pyspark $ make -f Makefile.pyspark-py36 $ make -f Makefile.java $ make -f Makefile.scala $ make -f Makefile.sparklyr $ make -f Makefile.pyspark-inc $ make -f Makefile.pyspark-py36-inc $ make -f Makefile.java-inc $ make -f Makefile.scala-inc $ make -f Makefile.sparklyr-inc
The default repository for the image can be overridden with the LOCAL_IMAGE var:
$ LOCAL_IMAGE=myimage make -f Makefile.pyspark
Modifying dependencies in the image yaml files
The cekit tool generates the image context directories based on the content of the image.*.yaml files. Please try to use the same version of cekit as noted in the current set of checked in Dockerfiles (currently 2.2.7)
A script has been provided to make altering the image.*.yaml files simpler. It handles modifying the specified versions of oshinko and Spark. Run this for more details
$ change-yaml.sh -h
Remaking image context directories when things change
The image context directories are generated with the cekit tool and contain the artifacts needed to build the images. They are:
* pyspark-build * pyspark-py36-build * java-build * scala-build * sparklyr-build * pyspark-build-inc * pyspark-py36-build-inc * java-build-inc * scala-build-inc * sparklyr-build-inc
If the yaml files used by cekit change (ie image.*.yaml) or the content included in an image changes (essentially anything under modules/), the image context directories need to be rebuilt.
Rebuilding context directories for an upstream pull request
If the changes being made are part of a PR to github.com/radanalyticsio/oshinko-s2i then all of the build directories should be generated from scratch. The best way to do this is with the make-build-dirs.sh script
$ make-build-dirs.sh
This will recreate the context directories starting from a clean environment, make sure any tarballs are truncated for github, and add all of the changes to the commit.
Rebuilding a particular context directory for testing/development
If the actual components specified in an image.*.yaml file have changed
as opposed to only the contents of existing modules, then the target
directory should be cleaned before generating the context directory
$ make clean-target
To regenerate a particular context directory, like pyspark-build, do this
$ make -f Makefile.pyspark clean-context context
To regenerate the context directory and also build the image, do this
$ make -f Makefile.pyspark clean build
Using incomplete images to choose an Apache Spark distribution
This link explains how to use radanalytics.io with a specific Spark distribution. It's a must read if you are not familiar with the incomplete images and how to use them.
Getting rad-image with R support enabled
The version of rad-image from radanalytics.io has generation of the R image
disabled. Do this to download rad-image and enable R support
$ ./get-rad-image.sh
Modifying templates/* with templates-is.sh
In addtion to resources.yaml from radanalytics.io, the templates in this repository
can be used with the imagestreams created by rad-image with a few changes.
Use the templates-is.sh script to generate modified templates in templates-is/
$ ./templates-is.sh
Run ./templates-is.sh -h for more information
Git pre-commit hook
The hooks/pre-commit hook can be installed in a local repo to
prevent commits with non-zero length tarballs in the image build
directories or to warn when changes have been made to yaml files or
scripts but the image build directories have not changed.
To install the hook locally do something like this:
$ cd .git/hooks $ ln -s ../../hooks/pre-commit pre-commit
This is recommended, since the CI tests will reject a pull request with non-zero length tarballs anyway. Save some time, install the hook.
Using release-templates.sh
The templates included in this repository always reference the latest https://hub.docker.com/u/radanalyticsio/. Those images may change during the normal course of development.
The release-templates.sh script can be used to create local versions of
the templates that reference s2i images from a particular oshinko release.
You may want to use this script to guarantee that you are using a stable image.
For example:
$ ./release-templates.sh v0.5.6 Successfully wrote templates to release_templates/ with version tag v0.5.6 grep radanalyticsio/radanalytics-.*:v0.5.6 * release_templates/javabuilddc.json: "name": "radanalyticsio/radanalytics-java-spark:v0.5.6" release_templates/javabuild.json: "name": "radanalyticsio/radanalytics-java-spark:v0.5.6" release_templates/python36builddc.json: "name": "radanalyticsio/radanalytics-pyspark-py36:v0.5.6" release_templates/python36build.json: "name": "radanalyticsio/radanalytics-pyspark-py36:v0.5.6" release_templates/pythonbuilddc.json: "name": "radanalyticsio/radanalytics-pyspark:v0.5.6" release_templates/pythonbuild.json: "name": "radanalyticsio/radanalytics-pyspark:v0.5.6" release_templates/scalabuilddc.json: "name": "radanalyticsio/radanalytics-scala-spark:v0.5.6" release_templates/scalabuild.json: "name": "radanalyticsio/radanalytics-scala-spark:v0.5.6" release_templates/sparklyrbuilddc.json: "name": "radanalyticsio/radanalytics-r-spark:v0.5.6" release_templates/sparklyrbuild.json: "name": "radanalyticsio/radanalytics-r-spark:v0.5.6" tar -czf oshinko_s2i_v0.5.6.tar.gz release_templates $ oc create -f release_templates/pythonbuilddc.json
MacOS Tips
For MacOS you will also need to download these tools: gsed and truncate. You can install these using homebrew and these commands:
brew install truncate
brew install gnu-sed
镜像拉取方式
您可以使用以下命令拉取该镜像。请将 <标签> 替换为具体的标签版本。如需查看所有可用标签版本，请访问 标签列表页面。
轩辕镜像加速拉取命令点我查看更多 radanalytics-r-spark 镜像标签
docker pull docker.xuanyuan.run/radanalyticsio/radanalytics-r-spark:<标签>
DockerHub 原生拉取命令
docker pull radanalyticsio/radanalytics-r-spark:<标签>
镜像拉取常见问题
功能
错误码
用户好评
来自真实用户的反馈，见证轩辕镜像的优质服务
oldzhang
运维工程师
Linux服务器
5
"Docker访问体验非常流畅，大镜像也能快速完成下载。"