How to use containers in data science with Docker and Azure: Part 1
In the first part of this introduction to containerisation, Jon Machtynger, Cloud Solution Architect for Advanced Analytics and AI at Microsoft, reveals what it can do for data scientists
Data science is a broad church, but there are common themes, and most practitioners have a general interest in how to operationalise data science processes. With this in mind, we’ve created a no-nonsense guide to getting with containerisation, which is increasingly being used by data scientists looking to standardise projects, and port easily from a local to a cloud environment.
In this containerisation primer I’ll be using Docker, an application that enables you to build, test and ship containerised applications from your PC or Mac. Other apps are available, but Docker is accessible, easy to install and is currently used by millions of developers.
Docker scales and provides resilience using orchestration capabilities such as Kubernetes. And it’s also relatively easy to move to the cloud, where it both scales and provides a mechanism for collaborating across many more people in a consistent fashion.
In this first instalment you will learn how to:
- Use a simple base image, build on it, and interact with it securely.
- Upload this to Azure and then remotely execute the same functionality there.
- Show how others might deploy that same functionality in their local environment.
We won’t cover the basics of how to install Docker, as you can find details on how to do this online, either via the official Docker support site, or elsewhere. I also assume that you already know the basics of what containers are, and how they differ from virtual machines (VMs).
There are many articles, such as About Docker, What is a Container, Installing Docker on Linux, Installing Docker on Windows, and Dockerfile Reference, that show how to install Docker, download containers or build a container.
But with this series, we will approach using Docker from a data science angle. And I’ll focus on the following data science inhibitors:
- Minimising conflicting library versions for different development projects.
- Consistency across environments and developers against specific design criteria.
- Avoiding a need to reinstall everything on new hardware after a refresh or failure.
- Maximise collaboration clarity across groups. Consistent libraries, result sets, etc.
- Extending on-premises autonomy/agility to cloud scale and reach.
We’ll interact with a container as though it was a separate, self-contained process you have complete access to, because that is – after all – what it is. This is a key concept to appreciate.
Getting started
Let’s start with some assumptions about working with containers:
- Containers are expected to be stateless and disposable. They provide compute and when they exit, everything inside that container could disappear with it. Because they’re stateless, they also provide a basis for scalability and resilience.
- A container should focus on as few things as possible. This supports a micro-services approach and allows designers to combine primitive containers in interesting combinations without overly worrying about interdependencies.
Building a simple Container
For the examples in this article, you should create a working directory. In that directory, we’re going to build a new container based on a very small image called alpine.
$ mkdir -p docker4ds $ cd docker4ds $ touch Dockerfile
Now edit that Dockerfile to hold the following:
FROM alpine
RUN apk --update add --no-cache openssh
RUN echo 'root:rootpwd' | chpasswd
# Modify sshd_config items to allow login
RUN sed -i 's/#PermitRootLogin.*/PermitRootLogin\ yes/' /etc/ssh/sshd_config && \
sed -ie 's/#Port 22/Port 22/g' /etc/ssh/sshd_config && \
sed -ri 's/#HostKey \/etc\/ssh\/ssh_host_key/HostKey \/etc\/ssh\/ssh_host_key/g' /etc/ssh/sshd_config && \
sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_rsa_key/HostKey \/etc\/ssh\/ssh_host_rsa_key/g' /etc/ssh/sshd_config && \
sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_dsa_key/HostKey \/etc\/ssh\/ssh_host_dsa_key/g' /etc/ssh/sshd_config && \
sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_ecdsa_key/HostKey \/etc\/ssh\/ssh_host_ecdsa_key/g' /etc/ssh/sshd_config && \
sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_ed25519_key/HostKey \/etc\/ssh\/ssh_host_ed25519_key/g' /etc/ssh/sshd_config
# Generate new keys
RUN /usr/bin/ssh-keygen -A && ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_key
CMD ["/usr/sbin/sshd","-D"] # Start the ssh daemon
This starts FROM a base Linux image called alpine, and then adds some custom functionality. I’m going to create a small SSH server, but in practice, I could add anything. The base image is pulled from the public Docker registry, but later I’ll show you how you can also pull your content from a private registry in Azure.
With the first RUN, I add the openssh package to the image and then assign a new password to root. Doing this in clear text isn’t secure practice, but I’m only showing the flexibility of a Dockerfile.
A Dockerfile can have many RUN steps, each of which, is an image build step that provides a layer of committed change to the eventual docker image. I then modify the sshd_config file to allow remote login and generate new SSH keys.
Lastly, I start the SSH daemon using CMD (CMD is the command that the container executes by default when you launch the build image, and a Dockerfile can only have one CMD).
You can find some Dockerfile best practices here.
Now let’s build the image:
$ docker build -t jon/alpine:1.0 . Sending build context to Docker daemon 109.1kB Step 1/6 : FROM alpine latest: Pulling from library/alpine bdf0201b3a05: Pull complete Digest: sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913 Status: Downloaded newer image for alpine:latest ---> cdf98d1859c1 Step 2/6 : RUN apk --update add --no-cache openssh ---> Running in d9aa96d42532 fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz (1/10) Installing openssh-keygen (7.9_p1-r4) (2/10) Installing ncurses-terminfo-base (6.1_p20190105-r0) (3/10) Installing ncurses-terminfo (6.1_p20190105-r0) (4/10) Installing ncurses-libs (6.1_p20190105-r0) (5/10) Installing libedit (20181209.3.1-r0) (6/10) Installing openssh-client (7.9_p1-r4) (7/10) Installing openssh-sftp-server (7.9_p1-r4) (8/10) Installing openssh-server-common (7.9_p1-r4) (9/10) Installing openssh-server (7.9_p1-r4) (10/10) Installing openssh (7.9_p1-r4) Executing busybox-1.29.3-r10.trigger OK: 17 MiB in 24 packages Removing intermediate container d9aa96d42532 ---> 1acd76f36c6b Step 3/6 : RUN echo 'root:rootpwd' | chpasswd ---> Running in 8e4a2d38bd60 chpasswd: password for 'root' changed Removing intermediate container 8e4a2d38bd60 ---> 4e26a17c921e Step 4/6 : RUN sed -i 's/#PermitRootLogin.*/PermitRootLogin\ yes/' /etc/ssh/sshd_config && sed -ie 's/#Port 22/Port 22/g' /etc/ssh/sshd_config && sed -ri 's/#HostKey \/etc\/ssh\/ssh_host_key/HostKey \/etc\/ssh\/ssh_host_key/g' /etc/ssh/sshd_config && sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_rsa_key/HostKey \/etc\/ssh\/ssh_host_rsa_key/g' /etc/ssh/sshd_config && sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_dsa_key/HostKey \/etc\/ssh\/ssh_host_dsa_key/g' /etc/ssh/sshd_config && sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_ecdsa_key/HostKey \/etc\/ssh\/ssh_host_ecdsa_key/g' /etc/ssh/sshd_config && sed -ir 's/#HostKey \/etc\/ssh\/ssh_host_ed25519_key/HostKey \/etc\/ssh\/ssh_host_ed25519_key/g' /etc/ssh/sshd_config ---> Running in 3c85a906e8cd Removing intermediate container 3c85a906e8cd ---> 116defd2d657 Step 5/6 : RUN /usr/bin/ssh-keygen -A && ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_key ---> Running in dba2ff14a17c ssh-keygen: generating new host keys: RSA DSA ECDSA ED25519 Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /etc/ssh/ssh_host_key. Your public key has been saved in /etc/ssh/ssh_host_key.pub. The key fingerprint is: SHA256:T4/Z8FLKLdEwZsTTIhx/g0A5DDuejhcl7B9FdsjFyCc root@dba2ff14a17c The key's randomart image is: +---[RSA 4096]----+ | .=+*+*o | | . .**Eo+ | | = .oO=o | | o = + = . | | = S + o | | o o = % | | . o . O = | | . o | | | +----[SHA256]-----+ Removing intermediate container dba2ff14a17c ---> 49ee4b262ae4 Step 6/6 : CMD ["/usr/sbin/sshd","-D"] # Start the ssh daemon ---> Running in 2a074ec11e30 Removing intermediate container 2a074ec11e30 ---> cf85e38faa5e Successfully built cf85e38faa5e Successfully tagged jon/alpine:1.0 $
If I now look at my available images, I have the core alpine image used as a basis for my image, and my custom image, which includes the SSH service. Look how tiny my image is – a working SSH server in under 13MB:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE jon/alpine 1.0 cf85e38faa5e 52 seconds ago 12.5MB alpine latest cdf98d1859c1 8 days ago 5.53MB
Notice that during the build of jon/alpine:1.0, the base alpine image had an ID of cdf98d1859c1, which also shows up in the Docker image lists. Let’s use it.
The following creates, and then runs a container based on the jon/alpine:1.0 image. It also maps port 2222 on my local machine to port 22 within the running container, which is the default SSH login port. When I create that container, it returns a unique identifier, and if I list running containers, it shows a unique container ID that prefixed that text:
$ docker run -d -p 2222:22 jon/alpine:1.0 db50da6f71ddeb69f1f3bdecc4b0a01c48fcda93f68ee21f2c14032e995d49ff $ $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES db50da6f71dd jon/alpine:1.0 "/usr/sbin/sshd -D" 5 minutes ago Up 5 minutes 0.0.0.0:2222->22/tcp obj_austin
I should now also be able to SSH into that container. I log in through port 2222, which maps to the container’s SSH port 22. The hostname for that container is the container ID:
$ ssh root@localhost -p 2222 The authenticity of host '[localhost]:2222 ([::1]:2222)' can't be established. ECDSA key fingerprint is SHA256:IxFIJ25detXF9HTc5CHffkO2DmhBzBe6EFRqFVj5H6w. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '[localhost]:2222' (ECDSA) to the list of known hosts. root@localhost's password: Welcome to Alpine! The Alpine Wiki contains a large amount of how-to guides and general information about administrating Alpine systems. See . You can setup the system with the command: setup-alpine You may change this message by editing /etc/motd. db50da6f71dd:~#
Multiple containers often work together owning different capabilities or scaling compute power by running in parallel. Let’s start more of these, each with a different mapped port:
$ docker run -d -p 2223:22 jon/alpine:1.0 5821d6a9e8c73ae0d64f7c59199a948cd43e87d6019b05ff54f01df83557b0f3 $ docker run -d -p 2224:22 jon/alpine:1.0 0305ea0aaf5142c6a89f8802fb67c3b1a768094a81be2bf15578b933c3385f87 $ docker run -d -p 2225:22 jon/alpine:1.0 1e0f3f2ac16f5fcd9a1bb169f07930061d42daea6aec8afeb08132ee5dd5c896 $ $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1e0f3f2ac16f jon/alpine:1.0 "/usr/sbin/sshd -D" 34 seconds ago Up 33 seconds 0.0.0.0:2225->22/tcp loving_kare 0305ea0aaf51 jon/alpine:1.0 "/usr/sbin/sshd -D" 39 seconds ago Up 38 seconds 0.0.0.0:2224->22/tcp nab_ritchie 5821d6a9e8c7 jon/alpine:1.0 "/usr/sbin/sshd -D" 44 seconds ago Up 42 seconds 0.0.0.0:2223->22/tcp det_feynman db50da6f71dd jon/alpine:1.0 "/usr/sbin/sshd -D" 12 minutes ago Up 12 minutes 0.0.0.0:2222->22/tcp obj_austin
After each container starts, it tells me its container ID. You can actually use any of a container’s initial uniquely identifying characters to work with it.
Let’s find out more about container 1e0f3f2ac16f using docker inspect 1e0. I’m interested in some network details. I only use the first three characters because they’re enough to uniquely identify it. In fact, I could have identified it with just ‘1’ as no other container ID started with that:
$ docker inspect 1e0 | grep -i address
"LinkLocalIPv6Address": "",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"GlobalIPv6Address": "",
"IPAddress": "172.17.0.5",
"MacAddress": "02:42:ac:11:00:05",
"IPAddress": "172.17.0.5",
"GlobalIPv6Address": "",
"MacAddress": "02:42:ac:11:00:05",
$
Now let’s interact with it directly:
$ docker exec -it 1e0 ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:05
inet addr:172.17.0.5 Bcast:172.17.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:21 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1558 (1.5 KiB) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
$
Here, I executed the ipconfig command interactively with the container identified (uniquely) by the identifier 1e0. Note that both approaches provided the same network and MAC address. Once the command finishes, control returns to your host. Let’s interact with another container and look at its network details using both approaches:
$ docker exec -it 030 ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:04
inet addr:172.17.0.4 Bcast:172.17.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:21 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1558 (1.5 KiB) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
$ docker inspect 030 | grep -i address
"LinkLocalIPv6Address": "",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"GlobalIPv6Address": "",
"IPAddress": "172.17.0.4",
"MacAddress": "02:42:ac:11:00:04",
"IPAddress": "172.17.0.4",
"GlobalIPv6Address": "",
"MacAddress": "02:42:ac:11:00:04",$
This shows that the second container is running, and shares a common Docker network.
Now let’s move this capability to the cloud. If you don’t have an Azure account, you can get a free trial here. You’ll also need to install the Azure CLI client, which you can get here.
Now let’s create a resource group to hold (and ringfence) all our resources. This will allow us to cleanly remove it all once we’re done playing with it.
I’m going to use the resource group docker-rg, so do the same, or choose one that works for you. I’m choosing ‘East US’ as my location. Feel free to choose one close to you, but for the purposes of this tutorial, it should make no difference:
$ az group create --name docker-rg --location eastus
{
"id": "/subscriptions//resourceGroups/docker-rg",
"location": "eastus",
"managedBy": null,
"name": "docker-rg",
"properties": {
"provisioningState": "Succeeded"
},
"tags": null,
"type": null
}
$
Within your resource group, you can now create an Azure container registry with the az acr create command. The container registry name must be unique within Azure. I’m using joncreg as my registry name, within my docker-rg resource group.
$ az acr create --resource-group docker-rg --name joncreg --sku Basic --admin-enabled true
{
"adminUserEnabled": true,
"creationDate": "2019-04-17T10:31:54.591280+00:00",
"id": "/subscriptions//resourceGroups/docker-rg/providers/Microsoft.ContainerRegistry/registries/joncreg",
"location": "eastus",
"loginServer": "joncreg.azurecr.io",
"name": "joncreg",
"networkRuleSet": null,
"provisioningState": "Succeeded",
"resourceGroup": "docker-rg",
"sku": {
"name": "Basic",
"tier": "Basic"
},
"status": null,
"storageAccount": null,
"tags": {},
"type": "Microsoft.ContainerRegistry/registries"
}
$
This provides you with quite a bit of information, but if you log in to your resource group, you can also find the name of your registry login server. We’ll use this later as we start storing and retrieving content from that repository:
$ az acr login --name joncreg Login Succeeded $ $ az acr show --name joncreg --query loginServer --output table Result ------------------ joncreg.azurecr.io $
Now let’s tag your local machine against that Azure login server. This will allow us to refer to and manage it as a cloud resource. Notice that the Image IDs for both your local image and the Azure image are currently identical:
$ docker tag jon/alpine:1.0 joncreg.azurecr.io/alpine:1.0 $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE joncreg.azurecr.io/alpine 1.0 cf85e38faa5e 10 hours ago 12.5MB jon/alpine 1.0 cf85e38faa5e 10 hours ago 12.5MB $
And now let’s push the tagged image out to the Azure Container Registry, and then confirm that the image is there:
$ docker push joncreg.azurecr.io/alpine:1.0 The push refers to repository [joncreg.azurecr.io/alpine] 06c6815029d6: Pushed 0b334f069f0f: Pushed 438f073b5999: Pushed 9d8531f069a1: Pushed a464c54f93a9: Pushed 1.0: digest: sha256:d5080107847050caa2b21124142c217c10b38c776b3ce3a6611acc4116dcabb0 size: 1362 $ $ az acr repository list --name joncreg --output table Result -------- alpine $ $ az acr repository show-tags --name joncreg --repository alpine --output table Result -------- 1.0 $
So, what can we do with this container in the cloud?
We can execute it, share it with others, or use it as a base for building other containers. But in order to access it, you’ll need to do it securely. Let’s create a service principal, which we’ll use to later. In this script, I add my container registry name (joncreg) and specify a unique service principal name (jon-acr-sp):
#!/bin/bash # Modify for your environment. # ACR_NAME: The name of your Azure Container Registry # SERVICE_PRINCIPAL_NAME: Must be unique within your AD tenant ACR_NAME="joncreg" SERVICE_PRINCIPAL_NAME=jon-acr-sp # Obtain the full registry ID for subsequent command args ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv) # Create the service principal with rights scoped to the registry. # Default permissions are for docker pull access. Modify the '--role' # argument value as desired: # acrpull: pull only # acrpush: push and pull # owner: push, pull, and assign roles SP_PASSWD=$(az ad sp create-for-rbac --name http://$SERVICE_PRINCIPAL_NAME --scopes $ACR_REGISTRY_ID --role acrpull --query password --output tsv) SP_APP_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv) # Output the service principal's credentials; use these in your services and # applications to authenticate to the container registry. echo "Service principal ID: $SP_APP_ID" echo "Service principal password: $SP_PASSWD"
When I run this, I get returned a secure principle ID and a password. I’ve obscured mine for obvious reasons, but you will need to remember these for later, so make a copy:
Service principal ID: 6xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx2 Service principal password: 6xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxf
Now let’s create and deploy a container, which I’ll call it alpine-ssh. It doesn’t do much, so I’ll only allocate a single virtual CPU and 100MB of RAM to it:
$ az container create --resource-group docker-rg --name alpine-ssh --image joncreg.azurecr.io/alpine:1.0 --cpu 1 --memory 0.1 --registry-login-server joncreg.azurecr.io --registry-username 6xxxxxxx-xxxx-xxxx-xxxb-xxxxxxxxxxx2 --registry-password 6xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxf --dns-name-label jm-alpine-ssh-314159 --ports 22
{
.
.
}
$
You should see a JSON response with loads of details confirming your deployment, but you can also find the fully qualified DNS name for your new container using the following:
$ az container show --resource-group docker-rg --name alpine-ssh --query ipAddress.fqdn "jm-alpine-ssh-314159.eastus.azurecontainer.io" $
It’s deployed. And I can now SSH to that address to log in:
$ ssh root@jm-alpine-ssh-314159.eastus.azurecontainer.io The authenticity of host 'jm-alpine-ssh-314159.eastus.azurecontainer.io (20.185.98.127)' can't be established. ECDSA key fingerprint is SHA256:IxFIJ25detXF9HTc5CHffkO2DmhBzBe6EFRqFVj5H6w. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'jm-alpine-ssh-314159.eastus.azurecontainer.io,20.185.98.127' (ECDSA) to the list of known hosts. root@jm-alpine-ssh-314159.eastus.azurecontainer.io's password: Welcome to Alpine! The Alpine Wiki contains a large amount of how-to guides and general information about administrating Alpine systems. See <http://wiki.alpinelinux.org/>. You can setup the system with the command: setup-alpine You may change this message by editing /etc/motd. wk-caas-edf9b7736da8406395657de1be9212b0-f3c593a7045e59fd38bbf0:~#
Simple, right? But I’d like to make it easy for other people to use this image as well. Stop and remove all your running container images:
$ docker ps -q 1e0f3f2ac16f 0305ea0aaf51 5821d6a9e8c7 db50da6f71dd $ $ RUNNING=$(docker ps -q) $ docker stop $RUNNING 1e0f3f2ac16f 0305ea0aaf51 5821d6a9e8c7 db50da6f71dd $ docker rm $RUNNING 1e0f3f2ac16f 0305ea0aaf51 5821d6a9e8c7 db50da6f71dd $
Lastly, remove the images holding your original SSH server:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE joncreg.azurecr.io/alpine 1.0 cf85e38faa5e 10 hours ago 12.5MB jon/alpine 1.0 cf85e38faa5e 10 hours ago 12.5MB $ $ docker rmi jon/alpine:1.0 $ docker rmi joncreg.azurecr.io/alpine:1.0
There is now no local copy of your Docker image, but if you still need to run it locally, then this isn’t a problem. You shouldn’t have to rebuild it. You – and others you allow – can simply get it directly from Azure. So, let’s run one – it will see that there isn’t one locally, download it, and then run the container locally:
$ docker run -d -p 2222:22 joncreg.azurecr.io/alpine:1.0 Unable to find image 'joncreg.azurecr.io/alpine:1.0' locally 1.0: Pulling from alpine bdf0201b3a05: Already exists 9cb9180e5bb6: Pull complete 6425579f73e9: Pull complete b7eda421926c: Pull complete 163c36e4f93a: Pull complete Digest: sha256:d5080107847050caa2b21124142c217c10b38c776b3ce3a6611acc4116dcabb0 Status: Downloaded newer image for joncreg.azurecr.io/alpine:1.0 65fef6f65c6c9fc766e13766d7db4316825170e7ff7923db824127ed78ad0970 $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 65fef6f65c6c joncreg.azurecr.io/alpine:1.0 "/usr/sbin/sshd -D" About a minute ago Up About a minute 0.0.0.0:2222->22/tcp gracious_khorana
We’ve pulled that image from your Azure container registry. If you get an error saying that you need to log in to your Azure Docker container environment, use the following (with your service principal username and password) and retry the previous docker run command:
$ az acr login --name joncreg --username 6xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx2 --pass 6xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxf Login Succeeded $
Note: Once you’ve finished, and you’ve created a specific resource group to test this, you might want to consider removing the resource group. It will also stop you being charged for things you’re no longer going to use. In my case, I did the following:
$ az group delete --name docker-rg
Conclusion
This brings the first phase of our project to an end. Using a standards-based cross platform environment (Docker), you created a virtual compute (Alpine Linux) resource, which was then moved to the cloud (we used Alpine, but you could do the same with other environments such as Ubuntu, CentOS, Windows, etc.).
We then made the image available to be shared as a standard environment, with the option to maintain multiple versions of it (e.g. 1.0, 1.1, 2.0 etc.). Assuming a user has permission, any Docker environment can pull and run that standard image. This means you can package a very specific combination of libraries, environments, and tools, to develop, test, and run in a consistent fashion.
Next time
In the next part of this series, we’ll look at data persistence using containers and how this can be provided locally and in the cloud. This starts to provide a foundation for something more usable from a data science perspective such as shared code, database access, and shared development frameworks.
Find out more
[msce_cta layout=”image_center” align=”center” linktype=”blue” imageurl=”http://approjects.co.za/?big=en-gb/industry/blog/wp-content/uploads/sites/22/2019/05/CLO19_azureKinectDK_024.jpg” linkurl=”https://azure.microsoft.com/en-gb/product-categories/containers/” linkscreenreadertext=”Accelerate your apps with containers” linktext=”Accelerate your apps with containers” imageid=”11460″ ][/msce_cta]
About the author
Jon is a Microsoft Cloud Solution Architect specialising in Advanced Analytics & Artificial Intelligence.
With over 30 years of experience in understanding, translating and delivering leading technology to the market. He currently focuses on a small number of global accounts helping align AI and Machine Learning capabilities with strategic initiatives. He moved to Microsoft from IBM where he was Cloud & Cognitive Technical Leader and an Executive IT Specialist.
Jon has been the Royal Academy of Engineering Visiting Professor for Artificial Intelligence and Cloud Innovation at Surrey University since 2016, where he lectures on various topics from machine learning, and design thinking to architectural thinking.
