I knew Microsoft’s cloud solution, Azure, supported Docker – but how well? It felt a little strange that Microsoft were embracing Docker so rapidly with Docker being built on Linux. This post is my first foray into trying to run Magento in a Docker image on Microsoft Azure. While I had a couple of speed bumps to work through (and a few cups of coffee waiting at times for things to compile/download), I must say I was pleasantly surprised with the tools.
I always knew Amazon Web Services (AWS) was #1, but I had not realized how popular Azure had become. Bloomberg’s Businessweek.com (http://www.businessweek.com/articles/2014-12-11/microsofts-azure-is-closing-the-gap-with-amazons-cloud-servIce) reports that Microsoft is closing the gap, with their market share estimate increasing from 7% to 10% with AWS decreasing from 28% to 27%. Sure, that is still a big gap, but it had not registered that Azure was in second position.
This, combined with noise around supporting Docker, was reason enough to go investigate further. Luckily for me, going to the home page http://azure.microsoft.com/ showed a ‘Free Trial’ link. Bonus! So I signed up for the free trial and dived in. (I also signed up with my credit card details, although I am not sure that was actually necessary or not.)
The Azure web based management dashboard is available from the “My Account” link on the home page (after you log on). One little hiccup I had in doing the steps below was that I did not initially appreciate I needed to allocate storage for my VM. To add new storage I clicked on the “New” button at the bottom of the page, then navigated to “Data Services” -> “Storage” -> “Quick Create”. For a quick create only 4 fields need to be entered:
- URL – in particular the DNS name to be created for the server. (I picked ‘akent’.)
- Location/Affinity Group – pick where you want the storage to be located.
- Subscription – I created storage with “Free Trial” as the choice.
- Replication – talks about which replication strategy you want in place. I picked Geo-redundant.
Azure CLI on Windows (or Linux)
I had never used Azure before so the first thing I did was a few Google searches of “Azure” and “Docker”. I quickly located a Cross Platform Command Line Interface (CLI) for talking to Azure nodes. I used the Windows installer described on the page.
The end result is ‘azure‘ is a new command I can run in my Windows environment from a command prompt. The command line utility is written using Node.js. I will say on my laptop the start up overhead per command is definitely noticeable. It’s a few seconds to even print a hello world message. But it worked pretty well. To work out all the command line options you can type azure help.
First thing I recommend doing is getting the authentication via certificates going. Azure makes this pretty easy.
> azure account download info: Executing command account download info: Launching browser to http://go.microsoft.com/fwlink/?LinkId=254432 help: Save the downloaded file, then execute the command help: account import <file> info: account download command OK
The command brings up a web page with instructions. It was pretty easy to save to local disk the ‘publishSettings’ file and then import it.
> azure account import "Pay-As-You-Go-Free Trial-12-14-2014-credentials.publishsettings" info: Executing command account import info: account import command OK
Once set up, it is easy to run azure commands from a command prompt without having to enter a username or password each time.
> azure account list info: Executing command account list data: Name Id Current data: ------------- ------------------------------- ------- data: Free Trial bf658129-420a-4eb8-b601-5992beb true data: Pay-As-You-Go 4f79d30e-d2d9-4f0e-b39f-3b1e7c1 false info: account list command OK
Docker VM on Azure
I had the azure command working – the next job was to create a VM to host Docker on. To do this, you need to select which image to use. Microsoft Open Technologies has a useful article describing how to best do this. Basically, request a list of all images then pipe it through grep to reduce the output. I chose to go with Ubuntu 14.04.
> azure vm image list | grep 14_04 b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_1-LTS-amd64-server-20141125-en-us-30GB
You might notice that the VM does not contain ‘Docker’ in the name. If I understand correctly this is because Azure can add Docker to an OS image afterwards. This reduces the list of images required (you don’t need variations of the same base OS with and without Docker installed). To create a new VM for hosting Docker images use the azure vm docker create command. So the above image is a standard Ubuntu image, to which Docker is added as a result of the ‘docker’ modifier.
Here is the full command I used to create the new VM. I exposed the SSH port (22) so I could log in, and supplied my account name (akent) and password. These can be entered interactively instead of on the command line.
> azure vm docker create -e 22 -l "West US" akent-ubuntu-docker "b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_1-LTS-amd64-server-20141125-en-us-30GB" akent SecretPassword.v1 info: Executing command vm docker create warn: --vm-size has not been specified. Defaulting to "Small". info: Found docker certificates. + Looking up image b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_1-LTS-amd64-server-20141125-en-us-30GB + Looking up cloud service + Retrieving storage accounts + Looking up cloud service warn: --location option will be ignored + Getting cloud service properties + Looking up deployment + Creating VM info: OK info: vm docker create command OK
Here is the end result:
> azure vm list info: Executing command vm list + Getting virtual machines data: Name Status Location DNS Name IP Address data: ------------------- ---------------- -------- -------------------------------- -------------- data: akent-ubuntu-docker RoleStateUnknown West US akent-ubuntu-docker.cloudapp.net 100.112.100.75 info: vm list command OK
I managed to log on to the VM directly (but soon discovered that it generally was not needed).
> ssh firstname.lastname@example.org email@example.com's password: ******** akent@akent-ubuntu-docker:~$ docker --version Docker version 1.4.0, build 4595d4f
Docker.exe on Windows
I run Windows on my laptop. As such I cannot run Docker containers on my laptop directly without using some form of virtualization software (like VirtualBox). However, I can develop and run a Magento site natively on my laptop then build a Docker image on my laptop. This is because Microsoft has contributed to the project with code allowing the Docker ‘build’ phase to be run natively on Windows. To do this however I had to compile up the Docker executable myself.
First step is to install Go (a programming language from Google). For windows I went to http://golang.org/dl/ and picked the MSI file from the list.
Next I used the following commands to compile up Docker based on https://ahmetalpbalkan.com/blog/compiling-docker-cli-on-windows/. I discovered the hard way it is important to use the directory names exactly as below.
> mkdir c:\go\src\github.com\docker > git clone https://github.com/docker/docker.git c:\go\src\github.com\docker\docker Cloning into 'c:\go\src\github.com\docker\docker'... remote: Counting objects: 65807, done. remote: Compressing objects: 100% (42/42), done. remote: Total 65807 (delta 13), reused 44 (delta 9) Receiving objects: 100% (65807/65807), 36.62 MiB | 449.00 KiB/s, done. Resolving deltas: 100% (43251/43251), done. Checking connectivity... done. Checking out files: 100% (1582/1582), done. > set GOPATH=c:\go;c:\go\src\github.com\docker\docker\vendor > set DOCKER_CLIENTONLY=1 > cd c:\go\src\github.com\docker\docker\docker > go build -v github.com/Sirupsen/logrus github.com/docker/docker/pkg/ioutils github.com/docker/docker/dockerversion github.com/docker/docker/pkg/promise ... github.com/docker/docker/graph github.com/docker/docker/api/client github.com/docker/docker/docker
The end result is a docker.exe executable. You may wish to ensure this executable is in your path.
Before starting up MySQL and Magento containers, lets consider a simple “Hello World” example. First, review what containers as loading. The Docker ‘ps -a’ command is how to see what is already running on the Azure VM. To connect to the remote host I used the -H command line option in conjunction with –tls. Initially there will be no containers created.
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
As well you can check what images are available locally. Initially for a fresh install no images will be available locally. Again, initially there are not images available.
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
The run the Busybox image, supplying a command of /bin/echo hi, use the following command. This will automatically cause it to be downloaded.
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls run busybox /bin/echo hi Unable to find image 'busybox:latest' locally busybox:latest: The image you are pulling has been verified 36ea3c5a: Pulling fs layer 46f9f060: Pulling fs layer a6c5b276: Pulling fs layer Status: Downloaded newer image for busybox:latest hi
If all is working proceed to the following steps to start up a MySQL container and a Magento web application container on the Azure cloud.
The following describes one way to create a new MySQL database instance. This will run in a separate container to the Magento web server application. As always, the -H and –tls command line options are used when connecting to a remote installation. If you ran the commands above you would see the busybox image is now available locally.
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE busybox latest e72ac664f4f0 10 weeks ago 2.433 MB
To start up the container, port 3306 needs to be opened up for access outside the MySQL instance in the VM. The first time you run command it will download the MySQL image for use.
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls run -d --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin mysql:5.6 Unable to find image 'mysql:5.6' locally mysql:5.6: The image you are pulling has been verified 07909bc5: Pulling fs layer b3b798be: Pulling fs layer c6f40cc3: Pulling fs layer ... 0e1054df: Pulling fs layer f592a52e: Pulling fs layer 0bbb442c: Pulling fs layer Status: Downloaded newer image for mysql:5.6 800b893dd1c42cc34b9d2de7242e202a45e2592735b5c7b890bdd4cb030329b7
Once up and running, you can inspect the log files of the container.
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls logs mysql 2014-12-15 06:41:16 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2014-12-15 06:41:16 13 [Note] InnoDB: Using atomics to ref count buffer pool pages ... 2014-12-15 06:41:37 1 [Note] Execution of init_file '/tmp/mysql-first-time.sql' started. 2014-12-15 06:41:37 1 [Note] Execution of init_file '/tmp/mysql-first-time.sql' ended. 2014-12-15 06:41:37 1 [Note] mysqld: ready for connections. Version: '5.6.22' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
Now we have a MySQL container up and running, we next need a Magento web server instance which will reference the MySQL instance.
Starting Magento 2
The following command will download and launch a Magento 2 Docker image on Magento 2. Again the Docker ‘logs’ command can we used to inspect the logs of the launched container. (see Magento 2 Demo on Docker and Panamax for more background on the following command.)
> docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls run -d --name magento2 -p 80:80 --link mysql:mysql -e MYSQL_USER=root -e MYSQL_PASSWORD=admin -e PUBLIC_HOST=akent-ubuntu-docker.cloudapp.net alankent/docker-magento2-git-demo:0.1.0-alpha108 cac50ea4f9e57210403d8168231da38e43e01360500626672793f3ee5faa83c4 > docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls logs magento2 + MYSQLAUTH='--user=root --password=admin' + mysql --user=root --password=admin -e '' + mysql --user=root --password=admin -e 'CREATE DATABASE IF NOT EXISTS magento' + cd /var/www/magento2/setup + php -f index.php install --cleanup_database --db_host=mysql --db_name=magento --db_user=root --db_pass=admin --backend _frontname=admin --base_url=http://akent-ubuntu-docker.cloudapp.net/ --language=en_US --timezone=America/Los_Angeles --c urrency=USD --admin_lastname=Smith --admin_firstname=John --firstname.lastname@example.org --admin_username=admin -- admin_password=admin123 --use_secure=0 Starting Magento installation: File permissions check... [Progress: 1 / 93] Enabling Maintenance Mode... [Progress: 2 / 93] Installing deployment configuration... [Progress: 3 / 93] Cleaning up database... Recreating database `magento` ...
Adding a port
My first attempt to connect to the web server on port 80 failed. The vm endpoint list command shows what ports each container has exposed outside of the container. Keeping this list of ports to a minimum reduces the risk of break-in for your store.
> azure vm endpoint list akent-ubuntu-docker info: Executing command vm endpoint list + Getting virtual machines data: Name Protocol Public Port Private Port Virtual IP EnableDirectServerReturn Load Balanced data: ------ -------- ----------- ------------ -------------- ------------------------ ------------- data: docker tcp 4243 4243 184.108.40.206 false No data: ssh tcp 22 22 220.127.116.11 false No info: vm endpoint list command OK > azure vm endpoint create akent-ubuntu-docker 80 info: Executing command vm endpoint create + Getting virtual machines + Reading network configuration + Updating network configuration info: vm endpoint create command OK > azure vm endpoint list akent-ubuntu-docker info: Executing command vm endpoint list + Getting virtual machines data: Name Protocol Public Port Private Port Virtual IP EnableDirectServerReturn Load Balanced data: --------- -------- ----------- ------------ -------------- ------------------------ ------------- data: docker tcp 4243 4243 18.104.22.168 false No data: ssh tcp 22 22 22.214.171.124 false No data: tcp-80-80 tcp 80 80 126.96.36.199 false No info: vm endpoint list command OK
The above may seem lengthy, but a lot of it relates to installing and configuring Azure. Once set up there are only a small number of commands that are required.
- Create a new VM with incorporated Docker support
- Create the MySQL instance
- Create the Magento 2 instance
- Expose port 80 of the Magento 2 instance
The commands (copied from the above) are as follows:
> azure vm docker create -e 22 -l "West US" akent-ubuntu-docker "b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_1-LTS-amd64-server-20141125-en-us-30GB" akent SecretPassword.v1 > docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls run -d --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin mysql:5.6 > docker -H tcp://akent-ubuntu-docker.cloudapp.net:4243 --tls run -d --name magento2 -p 80:80 --link mysql:mysql -e MYSQL_USER=root -e MYSQL_PASSWORD=admin -e PUBLIC_HOST=akent-ubuntu-docker.cloudapp.net alankent/docker-magento2-git-demo:0.1.0-alpha108 > azure vm endpoint create akent-ubuntu-docker 80
This can all be run from the command line on a Windows or Linux desktop or laptop.
Building container images to deploy is more complicated, but also possible from the Windows and Linux command lines now that the Docker executable (docker.exe) is available on both Windows and Linux. What this means is the development of a Docker container can be easily conducted on a laptop/desktop and pushed to Azure via command line commands. You don’t have to jump between environments. This makes automation straightforward as required.
It took me a little bit to get the tools working – mainly because I did not follow the instructions exactly the first time. When I went back and followed them more carefully things went much better. The end result however was with some relatively simple commands I could create a new VM and deploy a number of containers to the VM all from my laptop (I did not have to log on to the remote server due to the -H and –tls command line options of the Docker executable). The fact that it was so easy to get a VM up with Docker support, with appropriate authentication and certificates automatically deployed, impressed me. It felt pretty slick.
There is another aspect to all this however. Azure has suffered a few outages recently. I must admit I am not overly concerned by that. I am sure Microsoft are looking into it, and one of the main reasons I personally like Docker is it’s portability. If your hosting partner is not doing a good enough job, moving to a new hosting partner is much easier with Docker. The Docker images I used in this article have zero knowledge of Azure – the same image will happily run on Rackspace for example.
Tools in the Docker space are continuing to evolve. It would be great if more hosting partners make tools like Azure available where you can spin up a new Docker VM easily, then be able to push images to that VM all via relatively simple commands. Once in place, I think the norm for even small sites will have developers build and test a site on their laptop, then easily push the result to production in a hosted environment with confidence. The better Magento hosting partners will provide all the surrounding services (Varnish, Redis, database, and so on) allowing developers to focus on their site. This becomes particularly attractive to developers if the connections between services can be standardized.
The above is not a complete guide to Azure. But if you like Docker I do think its worth giving Azure a bit of a look. It was better than I anticipated.
http://www.cnbc.com/id/102285758?__source=yahoo%257cfinance%257cheadline%257cheadline%257cstory&par=yahoo&doc=102285758 – CNBC article on Microsoft azure and competition
Thanks for this nice post, it was very helpful!
A few notes that might be useful when using a windows 10 client:
1. openssl is required. To install it:
go to: https://code.google.com/p/openssl-for-windows/downloads/list
download “openssl-0.9.8k X64” and unzip
2. my “azure” command only works from an administrator command-prompt, not a standard one.
3. the account import needs a fully qualified filename:
azure account import “C:\Users\myname.mydomain\Downloads\Free Trial-8-20-2015-credentials.publishsettings”
4. run the “azure vm image list” without the grep
5. if needed (the docker client does not yet support windows 10):
copy all “*.pem” files from the “C:\Users\myname.mydomain\.docker” map to the “.docker” map on the machine that hosts the docker client.
6. on the docker client:
docker –tls –tlscacert=ca.pem –tlscert=cert.pem –tlskey=key.pem -H=myvm.cloudapp.net:2376 version
should give output like this:
API version: 1.20
Go version: go1.4.2
Git commit: d12ea79
Built: Thu Aug 13 02:35:49 UTC 2015