Skip to main content

Docker

Introduction

Docker provides operating-system-level virtualization, also known as containerization and refers to an operating system feature in which the kernel allows the existence of multiple isolated user-space instances. Such instances, called containers may look like real computers from the point of view of programs running in them. A computer program running on an ordinary person’s computer’s operating system can see all resources (connected devices, files and folders, network shares, CPU power, quantifiable hardware capabilities) of that computer. However, programs running inside a container can only see the container’s contents and devices assigned to the container.

By using Docker containers much of the complexity of installing and running software applications like Brytlyt can be significantly simplified.

Check the correct version of NVIDIA driver is installed

First, let’s check you have a cuda driver installed on machine. You can do it by running this:

nvidia-smi

The suggested driver is 384.XX or higher. If the driver is lower or you are not able to run this command, please re-install cuda drivers by following the instructions below.

First, remove the previous installation:

 sudo apt-get purge nvidia-*

Then run this:

sudo dpkg -i cuda-repo-__.deb 
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos///7fa2af80.pub 
sudo apt update 
sudo apt install cuda-drivers

For example:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo apt update
sudo apt install cuda-drivers

After these steps you should reboot your machine.

Install Docker

First, you need to have docker installed (following scripts comes from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce-1

sudo apt update

sudo apt install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable" 

sudo apt update

sudo apt install docker-ce

For IBM POWER architecture

Change arch to ppc64el

First, you need to have docker installed (following scripts comes from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce-1

sudo apt remove docker docker-engine docker.io

sudo apt update

sudo apt install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
   "deb [arch=ppc64el] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable" 

sudo apt update

sudo apt install docker-ce

For more architectures

Look here https://download.docker.com/linux/ubuntu/dists/xenial/InRelease

Installing Nvidia Docker

Now you can install nvidia-docker2 (following scripts comes from https://github.com/NVIDIA/nvidia-docker):

If there is Nvidia-docker v.1+ installed, it needs to be removed.

sudo docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge -y nvidia-docker

Add the package repositories

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update

Additional lists can be found here: https://nvidia.github.io/nvidia-docker/

Install nvidia-docker2, set default runtime as Nvidia and reload the Docker daemon configuration

sudo apt install -y nvidia-docker2
sudo tee /etc/docker/daemon.json &>/dev/null <<EOF
{
        "default-runtime": "nvidia",
        "runtimes": {
          "nvidia": {
              "path": "/usr/bin/nvidia-container-runtime",
              "runtimeArgs": []
          }
        }
}
EOF
sudo pkill -SIGHUP dockerd

Test nvidia-smi with the latest official CUDA image

sudo docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi

Pulling BrytlytDB docker image

You need to be logged in to your docker hub account. To do so, type:

sudo docker login

Then, to pull the image, type:

sudo docker pull brytlytdb/brytlytaws8

Running the BrytlytDB docker image

When you pulled the image, you can run the container:

sudo docker run -it--runtime=nvidia --name=my_name -p : --shm-size=32G \ --net=host brytlytdb/brytlytaws8:latest bash

 

-p : – this might be useful when you’d like to connect to the coordinator
outside the container,
–shm-size=N – this option is required if you want to dump/reload tables; the size is specified by user.

When you are in, you can start the BrytlytDB by typing:

 brytlyt start

Connecting to the BrytLytDB

Inside the container you can connect to BrytLytDB using this:

 psql -U brytlyt

 

You can also connect to theBrytLytDB outside the AWS machine:

 psql -p  -h  -U brytlyt