metasfresh/elk

About

Source: https://github.com/metasfresh/metasfresh-dev/tree/master/elk

This image contains an ELK-Stack with some custom configs and setting for a metasfresh docker-compose project.

The source image is built FROM sebp/elk:761 - https://www.github.com/spujadas/elk-docker

Enviroment Variables

These variables are applied to create a default policy to define index-rollover sizes and index-delete settings:

• ELK_INDEX_ROLLOVER_SIZE -> defines default policy for how big the current index shall be before creating a rollover (def.: 1G)
• ELK_INDEX_ROLLOVER_AGE -> defines default policy for how long the current index is stored before creating a rollover (def.: 30d)
• ELK_INDEX_DELETE_AGE -> defines how long indices, including rollovers shall be kept (def.: 7d)
• ELK_ELASTIC_HOSTNAME -> defines the hostname the settings above shall be applied to (def.: localhost)
• ELK_ELASTIC_JSON_PORT -> defines the port the settings above shall be applied to - JSON-Port (def.: 9200)

Docker-Compose Example

https://github.com/metasfresh/metasfresh-dev/blob/master/elk/docker-compose.example.yml

Connect metasfresh with ELK

1. modify your applications-logstash.properties of every application (app/webapi/etc.) you want to connect to the ELK-Stack

• add following lines

logstash.host=elk
logstash.port=5000

Where logstash.host is the hostname/IP of the elk container 2. rebuild your metasfresh-stack with the new application.properties 3. run the metasfresh java processed with -Dspring.profiles.active=logstash 4. connect to eg. Kibana using the exposed port of your Kibana web-interface (def. 5601)

DOCKERFILE

# source: https://www.github.com/spujadas/elk-docker
FROM sebp/elk:740

# to make sure that the cache is only used during one day, run docker build --build-arg CACHEBUST=$(date "+%Y-%m-%d")
# that way we should get the latest updates since the release of our base image 
# thx to https://github.com/moby/moby/issues/1996#issuecomment-185872769
ARG CACHEBUST=1

# update the system
RUN apt update --quiet --assume-yes \
 && apt upgrade --quiet --assume-yes

# install the APM server
#RUN curl -L -O https://artifacts.elastic.co/downloads/apm-server/apm-server-7.4.0-amd64.deb
#RUN dpkg -i apm-server-7.4.0-amd64.deb
#RUN update-rc.d apm-server defaults

COPY start.sh /usr/local/bin/start.sh
RUN chmod +x /usr/local/bin/start.sh
COPY elastic-post-hooks.sh /usr/local/bin/elastic-post-hooks.sh
RUN chmod +x /usr/local/bin/elastic-post-hooks.sh


# copy logstash config files
COPY 01-tcp-input.conf /etc/logstash/conf.d/01-tcp-input.conf
COPY 30-output.conf /etc/logstash/conf.d/30-output.conf

# set file mode to rw-r--r--
RUN chmod 0644 /etc/logstash/conf.d/01-tcp-input.conf \
 && chmod 0644 /etc/logstash/conf.d/30-output.conf

# ports in addition to the ones our basic image exposes
# port 5000 for logstash 
EXPOSE 5000