rvernica/apache-arrow

Apache Arrow

• Dockerfile to build https://arrow.apache.org/

• Build on top of https://www.debian.org/

• rvernica/apache-arrow Docker image builds:

  • arrow
  • parquet-cpp
  • pyarrow with feather, parquet and jemalloc

• rvernica/apache-arrow:pyarrow Docker image builds:

  • arrow
  • pyarrow with feather and jemalloc

Usage

The image comes with an example to verify that pyarrow and pyarrow.feather are working, see https://github.com/rvernica/docker-library/blob/master/apache-arrow/example.py:

python
>>> import pyarrow
>>> import pyarrow.feather
>>> import pandas
>>> import numpy

>>> pyarrow.from_pylist([1,2,3]) # doctest: +ELLIPSIS
<pyarrow.array.Int64Array object at 0x...>
[
  1,
  2,
  3
]

>>> fn = 'example.feather'
>>> df = pandas.DataFrame({'ints': numpy.random.randint(0, 10, 5)})
>>> pyarrow.feather.write_feather(df, fn)
>>> pyarrow.feather.read_feather(fn) == df
   ints
0  True
1  True
2  True
3  True
4  True

To test the example do:

bash
> docker run --tty --interactive rvernica/apache-arrow python example.py -v
...
9 passed and 0 failed.
Test passed.