clusterhq/powerstrip-flocker

powerstrip-flocker: portable volumes using just the docker cli

or: how I learned to love running stateful docker containers and stop worrying where the data is

!flying books to illustrate portable volumes

(Portable "volumes", hee hee.)

What's the problem?

When you want to run Docker in production, you want to run it across multiple machines and you probably want to use some orchestration tools. However, when you attach a volume to a Docker container, the machine it's running on becomes a pet when it should be cattle.

The solution

You should be able to run a stateful container with a given volume on any host in your cluster, and the platform should handle moving data around as necessary.

What is powerstrip-flocker?

powerstrip-flocker allows you to use the regular docker CLI commands to create or move flocker volumes, automatically moving volumes around between hosts in the cluster as-needed.

How does it work?

powerstrip-flocker connects the Docker Remote API to the new Flocker Volumes API via https://github.com/clusterhq/powerstrip.

Killer demo

Write some data into a volume on one host, read it from another:

$ ssh node1 docker run -v /flocker/demo:/data busybox sh -c "echo fish > /data/file"
$ ssh node2 docker run -v /flocker/demo:/data busybox sh -c "cat /data/file"
fish

Note that the volume above has become "global" to both hosts.

What this means

Finally you can run stateful containers in docker and stop worrying about where the data is.

In other words, powerstrip-flocker exposes a global namespace (/flocker/*) of volumes which flocker will move into place just-in-time before letting docker start your containers.

Orchestration

Because https://github.com/ClusterHQ/powerstrip-flocker presents a standard docker api - it means that the full gamut of orchestration tools can work with it:

• https://github.com/googlecloudplatform/kubernetes
• https://github.com/docker/swarm/
• https://github.com/mesosphere/marathon
• https://github.com/coreos/fleet

These tools don't have the concept of "portable volumes" - by leveraging the https://github.com/ClusterHQ/powerstrip API - https://github.com/ClusterHQ/powerstrip-flocker brings this concept to all of them for free.

Composition

Equally, https://github.com/ClusterHQ/powerstrip-flocker will work alongside other docker extension projects like networking tools:

• https://github.com/zettio/weave
• https://github.com/socketplane/socketplane
• https://github.com/Metaswitch/calico

The above tools "wrap" the docker cli and it was impossible to use them alongside other extensions like https://github.com/ClusterHQ/flocker. This is because both tools "wrap" the docker cli meaning only one can be used per container.

https://github.com/ClusterHQ/powerstrip allows the composition of multiple adapters. This means that a single docker host can now implement solutions for two of the critical issues which crop up when you run Docker on multiple hosts in production: storage and networking.

Is it ready yet?

This is a very early technology preview and has some https://github.com/ClusterHQ/powerstrip-flocker/issues. What's more, powerstrip itself is a prototype for the new Docker remote API extensions mechanism, and is therefore by definition unstable.

But, you can kick the tyres and try it today!

Then https://github.com/ClusterHQ/powerstrip-flocker/issues/new, to encourage us or tell us we're stupid.

OK, how do I try it?

We've put together a special demo preview environment:

You can try out our demo environment either with vagrant on local files, or on AWS.

Vagrant

This uses vagrant to bring up the demo on your local machine.

https://github.com/ClusterHQ/powerstrip-flocker/tree/master/vagrant

AWS

This uses vagrant to bring up the demo on AWS EC2 instances.

https://github.com/ClusterHQ/powerstrip-flocker/tree/master/vagrant-aws