A few people asked me recently on progress on “development environments”, which I am currently looking into. This is just a quick update because tweets are too short – especially for me!!  There is not much new content here over previous posts.

SDLC Phases

Let’s first talk about the typical phases in development – the Software Development Lifecycle.

You start a site from the core Magento code base. Not many use the default set of code – you beef up your site via customizations (e.g. designing a local theme), 3^rd party extensions, and more. This is what most people call “development” – doing whatever is needed to the code base to create the code base for the site, which is the output of this phase.

For advanced processes, a build phase may occur next. This is where the various assets required to construct a site are collected to create a build artifact. This might just be a copy of the source code for the site, but for a Docker based project it could be the Docker image.

Magento 2 is encouraging an increased usage of test automation. Some tests may be run before the build phase (e.g. typically unit tests are run first); other tests may be run on the build artifact itself, to ensure the build artifact itself is of good quality.

Finally you deploy the build artifact into production, upgrade the database schema, and turn on the upgraded code base.

SDLC Complexity

Magento is used by a wide range of users, from DIY merchants up to the largest system integrators. As such, there is not a single approach that fits all. However there does seem to be a degree of commonality.

• You develop the source code.
• You have the output artifact of the first stage.
• You deploy the artifact.

(Wow Alan! That is just soooo deeeep!)

What I mean by the above is there are several different strategies for developing code. Magento may recommend one approach, but there is no reason to exclude the other approaches. There is also then the “how do I get the code into production”. Some will have a simple approach (zip, ftp) and others will have a sophisticated CICD test and build pipeline, with automated rolling cluster deployments.

There can be mix and match – multiple approaches can be used to develop the site code; when done you pick one approach to get the code into production.

Let’s look at a few of the options here in greater detail.

Output Artifact

Let’s start at the middle of the above three. There are several approaches for the code artifact:

• You can make changes live in production. Add a theme, adjust a CSS file, make a configuration file change. Making changes live has higher risk, but very low overheads. This approach is more attractive for sites where the cost of things going wrong is not very high. So the artifact is just the code on production.
• Alternatively, the source code could be bundled up in a ZIP file or similar, to make moving from a development laptop to a production server easier.
• Another common approach is to put the source code into a source code control system, such as Git or SVN.

The deploy-to-production phase has a few options:

• Magento Cloud! (Well, I had to get that in here somewhere don’t I?) You commit the code to Git which triggers an automatic build of an image that is then deployed into an environment based on the Git branch name.
• Advanced solution partners would weave in test automation runner frameworks, like Jenkins or Travis.
• A Docker based deployment strategy would build an image (or images), test them, and roll out into production. The important aspect here is the same Docker image is both tested and deployed.
• Depending again on SDLC maturity levels, the new deployment may be first rolled out to a test or staging environment before the rollout to production. This step is important for sites with low risk tolerance. You want to try the deployment (including code and database upgrades) before attempting them in production, to minimize risk.

Backing up to the first phase again, the development phase.

• Even for very simple SDLC approaches, there are basic rules you want followed, such as “don’t edit the core!” Create a new theme if you want to customize the provided HTML, CSS, or JavaScript files.
• For simple sites where the changes are made directly in production, there is no real need for a separate development environment. You just use production as your development environment. Spin up a GoDaddy or similar instance, add a few extensions directly to the live store, do some simple local edits, maybe create a local theme, but no need for Git or similar to manage source code. Version control is to take a backup of the source code periodically, or copy files to “.bak” before you start editing.

If you want to develop a site on your laptop, then push to production hardware later, there are a few options:

• Go native. Get all the required software installed directly on your laptop. Great performance, great control, quite a bit of software to install to make sure you have the right versions of everything.
• Be Vagrant. Be able to move the same set of tools to different environments (nowhere is truly your real home). The benefit of Vagrant is one person picks the set of versions of tools to use, then everyone else can share the setup. Great for standardization, and built in support in PHP Storm (common PHP IDE).
• Mimic Vagrant using Docker. Vagrant creates one VM with everything installed, you can do the same thing with Docker. (Many of my recent personal projects have been doing this.)
• Real Docker. The idea behind Docker is to have standard, small, containers that you can mix and match. For example, you can have an NginX or Apache container, and use the one you want. Someone works out a good Redis configuration – everyone else then just uses it. The theory is also you can use the same image in development as production, but a “good” image for production locks down the image and has it rather skinny; this is not good for development as it means less tools are installed. I am a little less convinced in this approach as a “good” approach for being developer friendly for the masses – I found it rather complicated with less good developer tools being available. But there is the definite advantage that getting from dev to prod is typically smoother (you can use the same image on dev and prod, just with different resources allocated).
• In the cloud. There are cloud based development environments, such as CloudAnywhere I have mentioned in other posts. Avoids the need for local setup.

Conclusions

So what have I been up to, the real topic for this quick note. I have been trying several of the above approaches, with current thoughts and feelings.

• Native – too painful for the masses.
• Vagrant – I think a sensible default, just want to find a good box to use.
• Docker mimicking Vagrant – has some pros and cons compared to Vagrant. I think in a few months’ time with the release of the next version of Docker, Docker is going to be able to do what Vagrant can but faster. There is more momentum in Docker from what I see.
• Docker with many containers – seems great for production, but overly complex for development if you don’t need the “I want to use production images on my development box, for extra testing”.

I have been building up a few options to get firsthand experience. Vagrant has work by Joshua Warren as well as the Magento team Vagrant box; I have built a few mimic-Vagrant-using-Docker images (I like them personally); then there are a few multiple container Docker based Magento deployments out there.

The problem I keep coming back to is file system access. I want access to files from PHP Storm running natively on my laptop. I want easy to use, fast, with Grunt/Gulp support so saves to disk of Less files are detected automatically to trigger CSS compilation with auto-reloading in the web browser. The next Docker release should make this very slick on Mac’s. Vagrant I have been using rsync-auto so I save on my Windows laptop and it auto-pushes into the container. On Docker I either use an rsync like approach, or use Samba so I can mount the drive directly and see all generated files.

So what is next? I am interested to collect a bit more feedback from developers. I also want to build a real (mini) project to experience the tools first hand. I think Vagrant has mindshare today, but Docker will catch up quickly once the next release is out. I want to play a bit more to really use the different tools. For example, with Docker I find the network goes crazy every so often requiring a reboot; with Vagrant I find aborting a “vagrant rsync-auto” command sometimes leaves it running in the background. None of the tools are prefect.

So no answer from me yet – but probably should have one before the end of May. This post was just a brain dump / update. Separating the development environment from the source code artifact does have the advantage that swapping later is not difficult. Right now I am leaning Vagrant because of the greater share today. But really it does not matter that much if the Git repo structure can be standardized, because the development environment is unrelated to the deployment pipeline.