wangxiang5050/datax

镜像概述和主要用途

该镜像作为datax-executor的基础镜像，集成了datax工具及其运行所需的java、mvn、python等依赖环境，支持通过不同版本扩展各类数据读写插件，满足多样化的数据同步需求。

核心功能和特性

• 集成datax工具，提供数据同步能力
• 包含java 1.8.0_222、mvn 3.6.3、python 3.6.9等基础依赖
• 各版本持续新增插件，如hbase、influxdbreader、elasticsearchwriter等
• v1.0.1-hbase版本修复python3与datax py文件的兼容性问题

使用场景和适用范围

适用于需要运行datax数据同步任务的场景，特别是需要使用hbase、influxdb、elasticsearch等数据源进行数据读写的业务需求，可作为datax-executor的基础环境快速部署。

版本说明

v1.0.4-hbase

• 新增功能：增加influxdbreader插件
• Dockerfile：

Dockerfile
FROM wangxiang5050/datax:v1.0.4
COPY ./hbase11xsqlwriter /datax/plugin/writer/hbase11xsqlwriter

v1.0.4

• 新增功能：增加influxdbreader插件
• Dockerfile：

Dockerfile
FROM wangxiang5050/datax:v1.0.3
COPY ./influxdbreader /datax/plugin/reader/influxdbreader

v1.0.3

• 新增功能：增加elasticsearch插件
• Dockerfile：

Dockerfile
FROM wangxiang5050/datax:v1.0.2-hbase
COPY ./elasticsearchwriter /datax/plugin/writer/elasticsearchwriter

v1.0.1-hbase

• 新增功能：增加hbase插件；替换/datax/*.py文件，适配python3
• Dockerfile：

FROM wangxiang5050/datax:v1.0.0
COPY ./hbase20xsqlwriter /datax/plugin/writer/hbase20xsqlwriter
COPY ./hbase20xsqlreader /datax/plugin/reader/hbase20xsqlreader
COPY ./*.py /datax/bin/

v1.0.0 (已弃用)

• 包含软件：java 1.8.0_222、mvn 3.6.3、python 3.6.9、alibaba datax（路径：/datax）
• 弃用原因：python3与/datax/*.py文件不兼容，该问题在v1.0.1-hbase版本中修复
• Dockerfile：

### 1. Get Linux
FROM alpine:3.7

### 2. Get Java via the package manager
RUN apk update \
&& apk upgrade \
&& apk add --no-cache bash \
&& apk add --no-cache --virtual=build-dependencies unzip \
&& apk add --no-cache curl \
&& apk add --no-cache openjdk8-jre

### 3. Get Python, PIP

RUN apk add --no-cache python3 \
&& python3 -m ensurepip \
&& pip3 install --upgrade pip setuptools \
&& rm -r /usr/lib/python*/ensurepip && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache

### Get Flask for the app
RUN pip install --trusted-host pypi.python.org flask

####
#### OPTIONAL : 4. SET JAVA_HOME environment variable, uncomment the line below if you need it

ENV JAVA_HOME="/usr/lib/jvm/java-1.8-openjdk"

####

# Downloading and installing Maven
# 1- Define a constant with the version of maven you want to install
ARG MAVEN_VERSION=3.6.3 

# 2- Define a constant with the working directory
ARG USER_HOME_DIR="/root"

# 3- Define the SHA key to validate the maven download
# ARG SHA=b4880fb7a3d81edd190a029440cdf17f308621af68475a4fe976296e71ff4a4b546dd6d8a58aaafba334d309cc11e638c52808a4b0e818fc0fd544226d952544

# 4- Define the URL where maven can be downloaded from
ARG BASE_URL=[***]{MAVEN_VERSION}/binaries

# 5- Create the directories, download maven, validate the download, install it, remove downloaded file and set links
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
  && echo "Downlaoding maven" \
  && curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
  && echo "Unziping maven" \
  && tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
  \
  && echo "Cleaning and setting links" \
  && rm -f /tmp/apache-maven.tar.gz \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

# 6- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located

ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
EXPOSE 81    

# install datax
RUN wget [***] -P / \
  && cd / \
  && tar xvf datax.tar.gz \
  && rm -f /datax.tar.gz \
CMD ["date"]