Docker¶
The easiest way to run Crail is to use Docker images. We provide two preconfigured Docker images:
- TCP/DRAM: apache/crail
- RDMA/DRAM: apache/crail_rdma
If you want to run other or more complex configurations you can use either image as a basis and provide your own configuration files. Refer to Own configurations for details.
- and 2. share the following configuration parameters:
Property | Default Value | Description |
---|---|---|
NAMENODE_HOST |
localhost | Namenode hostname/ip to bind to |
NAMENODE_PORT |
9060 | Namenode port to listen on |
INTERFACE |
eth0 | Datanode network interface to listen on |
DATAPATH |
/dev/hugepages/data | Datanode hugepage path to data |
STORAGELIMIT |
1073741824 | Datanode Size (Bytes) of DRAM to provide |
CACHEPATH |
/dev/hugepages/cache | Client/Datanode hugepage path to buffer cache |
CACHELIMIT |
0 | Size (Bytes) of hugepage buffer cache |
These properties can be specified as environment variables when
starting a Docker image with -e <property>=<value>
.
TCP image¶
To run Crail you first need to start the namenode. For example
docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 apache/crail namenode
starts a TCP namenode using Docker’s host network configuration on host02 on interface eth5. The TCP tier allows for other network configurations. Refer to https://docs.docker.com/network/ for details.
Once the namenode has started start a Crail TCP storage tier (preferably on a different node). For example
docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 apache/crail datanode
starts a TCP datanode with 1GB of storage (default) listening on eth5. It is recommended to put the data directory on a hugetlb mount. You can do this by passing an mounted hugetlb directory on the host as a volume to Docker. For example
docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 -v /dev/hugepages:/dev/hugepages apache/crail datanode
passes the hugetlb mount /dev/hugepages to the container.
RDMA image¶
To run Crail/RDMA/DRAM you can start the namenode as follows:
docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 --cap-add=IPC_LOCK --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm -v /dev/hugepages:/dev/hugepages apache/crail_rdma namenode
This starts a namenode on host02 using the host’s network on eth5. Note that the uverbs device needs to match the interface.
To run a RDMA storage tier:
docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 --cap-add=IPC_LOCK --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm -v /dev/hugepages:/dev/hugepages apache/crail_rdma datanode
Note: The RDMA docker image provides default RDMA provider libraries from Ubuntu 18.04. They might not be compatible with your host’s RDMA stack. To install your own RDMA libraries use -v or create your own Docker image from crail_rdma.
Own configurations¶
If you want to run more complex configurations that are not covered by the options above you have two options:
- Create your own Docker image by creating a Dockerfile and using the crail images as a source. You can then change the configuration by e.g. copying your own config into the image
- Pass your config as a volume with
-v <local_path>:<docker_path>