Noodlings on Docker for Development in Magento 2 (Quick Note)

If you follow my blog, you will have seen I was doing some recent exploration of Gulp. You may also know I am quite interested in Docker. So it was interesting to try them out together.

For Docker, you can have files located in the container. But I frequently blow the container away and start again if it goes “strange” (or just to reset to a known state). If the master files are inside the container, you lose your work. Ugg.

Docker has the concept of volumes, which are useful. You can have directories of the host OS mounted inside the container. Deleting the container does not lose the files in the host mounted directories.

But, in addition, I work on Windows (yeah, yeah, I know! I have to do lots of PowerPoint decks and Outlook stuff, so until Microsoft does a better job on OS X I am sticking with Windows). This means I run Docker inside a VM. That means if I lose the VM instance I lose my files again. So I not only want volumes, I want them going all the way back to the host OS (through the VM layer).

And this is where the reports of slowness come in. VirtualBox for example is known to be somewhat slow for sharing Windows/OS X files system with the Docker running in the VM. Some approaches have improved performance (e.g. using NFS), but there are less options here for Windows than OS X.

For Windows, I was getting reasonable performance for having app/code, app/design, and app/i18n mounted but everything else living inside the container. Not perfect, but it was working pretty well and my files were protected against loss.

(Oh, I am quite liking the simplicity of installation for the latest “Docker Tools” available from Docker. Kitematic (alpha) is a little buggy still, but the installation was a breeze. Volume mounting is also pretty easy – just a few clicks in a GUI and one.)

Then I hit my next problem. Gulp works by using inotify to watch for file system changes. On file change, it automatically triggers a Less compilation build. Combined also with Live Reload, this greatly helps front end devs. Well, of course it would be too easy if Windows/OS X files system inotify events were exported through VirtualBox. So if you mount app/design this way, Gulp does not spot the file system changes. Bummer.

So more recently I have been exploring a few text editors to see how good a job of file syncing over say SFTP. (I picked SFTP because it was easy to install inside Docker and easy to authenticate – no messing with keys. This might not be a good long term solution.)

For example, editors that seem to support SFTP syncing include

  • PHP Storm – but only after fiddling with the server side configuration to accept the protocol versions PHP Storm uses, and pushing ‘vendor’ killed it (exponential slow down with number of files), so back to just pushing selected files or directories.
  • Atom, by GitHub – but I could not actually get it to work yet.
  • Sublime Text – I have not actually tried it yet, but numerous people said “it’s good”.
  • Bluefish – but I could not get it to work either.

Hmmmm. It was not the easy setup and configuration experience I was hoping for.

But the file system sync approach seems to have the following benefits:

  • Inside the container things run fast – all local file system.
  • The source files are protected – the master copy is not inside the container.
  • Gulp works – the push into the container generates inotify events correctly.
  • You can use composer inside the container to download everything into the ‘vendor’ directory, then just have the “real” project files pushed over on top (one-way sync).
  • It’s easy to put all the files you change under source code management without committing all of Magento – you commit the local files only.

So I am starting to warm up to the “sync files from local OS into the Docker image” approach, even though the setup of text editors seemed a bit harder than I expected.

That led me to browsing around and finding tools like lsyncd. That way, you can use *any* tools or editors modifying the local file system and have changes efficiently pushed into the Docker container (in one direction). It would be nice to have an easy to install tool for Windows and OS X that was easy to configure to sync to the Docker image. (lsyncd looks good, but is provided as source code – so Magento developers first need to get a C compiler installed? Not exactly a frictionless install experience!)

[Update: Feb 4, 2016: WinSCP seems to have a “continuously watch and copy” mode (Ctl-U) that looks pretty close to lsyncd.]

So how about “install vanilla Magento inside the Docker container, then do all customizations in your local file system and use an inotify based approach to efficiently sync file system changes into the Docker container”. With this approach, sometimes you do log in and do things inside the container, but most of the time you are editing source code locally. You then put only the local files in Git. The web server however runs inside the Docker container making it easy to share configuration between developers.

Do you use an inotify (or similar) based file sync scheme for your projects? Any experiences you want to share? Any “easy to set up perfect tool” that you use to sync files from Windows and OS X to Linux?


  1. Alan – adds NFS file sync to the volume so it is as fast as native file changes. Trust me, you’re overcomplicating things 🙂

    1. Nfs file sync works well for Mac, but I want it working well for Windows as well. Do you have that working as well? Happy to “steal” your approach and merge it in here…

      Trying samba inside container at the moment (mounting files in container as a Windows network drive), but PHP Storm takes ages to complete sync and “local file watch” turns off because it’s across a network mount.

      It feels like there should be something easy in all these options, if you have one working for Mac and Windows (or even just Mac – maybe have different recommendations for both platforms) I am all ears! 😉

  2. Also, I use a container that installs magento, and runs a composer update to install the modules, including my theme (from git.)

    So once I have tested my docker container locally, I build and “deploy” a new container in production. The magento container is fully disposable.

    1. Do you have a blog post or similar to link to from here?

  3. designhauser · · Reply

    Thanks for your insights, Alan. I go with NFS sync also on Mac. That’s working pretty fine.

    But did you find a solution to build the container for a production test? I imagine setting the magemode to production, running compilation and so on within my PHP-FPM service. But that way I would have to setup a database and run the magento installer everytime – if done on a separate build server. Compilation seems to be only possible if magento is up and running. So simply pushing composer.json, composer.lock and Dockerfile to the build server might not be possible. Any ideas on this?

    1. I saw some notes today indicating the use of DB for themes has been tracked down to no longer be required – so we will remove that code to avoid the DB dependency at build time (I think it should work)

      1. designhauser · ·

        Wow, that’s a real improvement! Thanks.

  4. I had the problem to support Magento 1 and 2 at the same time on Mac OS. That was really hard because of different MySQL and PHP versions. Luckily I got to docker and build a minimal environment for Magento 2. I also experience the poor performance of vboxsf (shared folder) which made it impossible to develop. I than found out more about gulp which I actually use now for syncing files to the container. There is also a task for copying all files from the container back to the local file system. I made it available on github and wrote a tutorial for that, if anyone is interested:

  5. You mentioned lsyncd. Did you manage to get that working? If yes, would be interesting to hear about it. Because, according to what I have read around, Virtualbox does not emit any inotify events if files are changed over mounts. So, in your example, lsyncd will not know when files in app/code or app/design have changed.

    1. I have not spent any more time with lsyncd sorry. I have been using Unison recently with some success. I had to find the same version of the binary on multiple platforms (the biggest issue) but it seems to work pretty well otherwise.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: