Skip to main content

Docker

Introduction

Docker provides operating-system-level virtualization, also known as containerization and refers to an operating system feature in which the kernel allows the existence of multiple isolated user-space instances. Such instances, called containers may look like real computers from the point of view of programs running in them. A computer program running on an ordinary person’s computer’s operating system can see all resources (connected devices, files and folders, network shares, CPU power, quantifiable hardware capabilities) of that computer. However, programs running inside a container can only see the container’s contents and devices assigned to the container.

By using Docker containers much of the complexity of installing and running software applications like Brytlyt can be significantly simplified.

Check the correct version of NVIDIA driver is installed

First, let’s check you have a cuda driver installed on machine. You can do it by running this:

nvidia-smi

The suggested driver is 384.XX or higher. If the driver is lower or you are not able to run this command, please re-install cuda drivers by following the instructions below.

First, remove the previous installation:

 sudo apt-get purge nvidia-*

Then run this:

 sudo dpkg -i cuda-repo-__.deb sudo apt-key adv --fetch-keys https://developer.download.nvidia.com
/compute/cuda/repos///7fa2af80.pub sudo apt-get update sudo apt-get install cuda-drivers

For example:

sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
sudo apt-key adv --fetch-keys
http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7f
a2af80.pub
sudo apt-get update
sudo apt-get install cuda-drivers

After these steps you should reboot your machine.

Install Docker

First, you need to have docker installed (following scripts comes from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce-1

sudo apt-get update

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable" 

sudo apt-get update

sudo apt-get install docker-ce

For IBM POWER architecture

Change arch to ppc64el

First, you need to have docker installed (following scripts comes from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce-1

sudo apt-get remove docker docker-engine docker.io

sudo apt-get update

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
   "deb [arch=ppc64el] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable" 

sudo apt-get update

sudo apt-get install docker-ce

For more architectures

Look here https://download.docker.com/linux/ubuntu/dists/xenial/InRelease

Installing Nvidia Docker

Now you can install nvidia-docker (following scripts comes from https://github.com/NVIDIA/nvidia-docker):

We are currently supporting Nvidia-docker v.1.0.1 and if there is v.2+ installed, it needs to be removed/replaced.

sudo apt-get purge -y nvidia-docker2

Add the package repositories for amd64

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/amd64/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update

Add the package repositories for POWER ppc64

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update

Additional lists can be found here: https://nvidia.github.io/nvidia-docker/

Install nvidia-docker and reload the Docker daemon configuration

sudo apt-get install -y nvidia-docker
sudo pkill -SIGHUP dockerd
sudo apt-get install nvidia-modprobe
sudo service nvidia-docker start

Test nvidia-smi with the latest official CUDA image

sudo nvidia-docker run --rm nvidia/cuda nvidia-smi

Pulling BrytlytDB docker image

You need to be logged in to your docker hub account. To do so, type:

sudo docker login

Then, to pull the image, type:

 sudo docker pull brytlytdb/brytlytaws8

Running the BrytlytDB docker image

When you pulled the image, you can run the container:

 sudo nvidia-docker run -it --name=my_name -p :
--shm-size=32G --net=host brytlytdb/brytlytaws8:latest bash

 

-p : – this might be useful when you’d like to connect to the coordinator
outside the container,
–shm-size=N – this option is required if you want to dump/reload tables; the size is specified by user.

When you are in, you can start the BrytlytDB by typing:

 brytlyt start

Connecting to the Brytlyt coordinator

Inside the container you can connect to coordinator using this:

 psql -U brytlyt

 

You can also connect to the coordinator outside the AWS machine:

 psql -p  -h  -U brytlyt