If you have been following my blog, you will know I have been trying out a number of different development environment options. The goal is to work out a standard Magento offering or offerings to have as a default. This blog post is my next stage of evolution in this process, and as always making it public in case other interested parties want to also experiment, contribute, or comment (see “magento-dev”).

The thrust of this post is to talk about the different stages of the lifecycle of a project, providing more standardization and consistency, while still allowing the diversity that is essential for such a broad ecosystem.

Project Source Code Structure

Standardization of project source code directory structures is useful for both humans moving between projects and tools. I still advocate code local to the project going under app/code and app/design, with code external to the project being installed in vendor. The vendor directory is not normally committed to source code control. An important concept still, but I am not proposing any changes here from previous posts.

Source Code Management

Personally, I would never do a project without source code management. However, for simple sites it may be decided to just take periodic backups of the source code instead. So source code management tools such as git and svn should work with the rest of the tools described here, but should not be mandatory.

Environmental Technologies

I have no objection to developers using native tools, but I find virtualization technologies like Vagrant and Docker do offer an easy way to build Magento specific builds of development tools. Magento internally is unlikely to offer too many alternatives here, but mandating one approach for everyone to follow appears unrealistic. Tools evolve over time. So I am currently leaning towards one (or more) standard approaches from Magento, while making sure other environments can be just as easily used.

For example, are you on a Mac with Docker 1.12 installed? Great, use its superfast shared native file system support sharing support. On Windows with Vagrant? Great, use rsync to copy files from the native file system into the container. A vi hacker? Great, maybe you could just edit the files inside the container natively. Frontend developer? Pick Grunt or Gulp compatible solutions. I don’t think there should be too many promoted options here, but picking one to the exclusion of other choices feels like a mistake. But make some good general default options easily accessible to the general public – they can find other more finely tuned solutions themselves later.

Environment Configuration Files

Files needed to set up a local environment (such as Vagrantfile and docker-compose.yml) I am advocating to avoid committing into the project source code repository. (At least it should not be mandatory.) Instead, treat such files as specific to a developer’s local environment. One of the tricks here is to not accidentally commit such files into the source code repository. The .gitignore file is useful here.

Another issue (described below) is to avoid copying such files to the production environment. They are unlikely to break anything, but it is cleaner to not push them to production.

Pulling Code

Sometimes a project will start with a spun up site (e.g. from a single click install on a hosting provider), with the next step to then pull that code down to a development machine. Other times the master source code will be in a source code repository such as git. Other times a new project will be started from an empty slate. In all cases it should be easy to grab a copy of the code and start local development. But if a site was spun up on a hosting partner, make sure it is easy to get that copy of the code to capture any configuration settings that have already been set up for that provider.

(Yes, some merchants prefer to make changes on the site live in production. Turn on maintenance at night, add an extension, turn the site back on. It is a Magento product requirement to support this mode of operation. My personal goal is to make it so easy to have a local development environment, the norm would be to pull the code to a local development environment to avoid direct manipulation of your production site. But step one is to make it easy to do.)

Deploying to Production

After changes are made in a local development environment, tests all pass, etc, the next stage is to get your code into a staging and ultimately production environment. Automation here is highly desirable as it reduces the chance of human error during deployment. Ideally you code additional checks and gates into the deployment process to take the place of manual checks.

Magento-dev

“magento-dev” is a little command line script I am experimenting with to try out some of the above concepts. I am not saying this code will become production code – it is just a prototype. So if you go have a look, sure I should use the Symfony Console, real command line parsing, better indentation, tabs instead of spaces. If it bugs you that much, submit a pull request! 😉 If the code runs, I can try out the experience. My goal is just to have a working prototype and learn.

The commands currently are as follows:

magento-dev create <environment>

Creates a new Vagrantfile, Dockerfile, whatever, based on the environment name. First cut I have created vagrant-rsync which steals the Vagrantfile I have been trying from my Alexa integration project (with some slight improvements).The idea is you keep your master source code native on your laptop, then use rsync to copy changes into the Vagrant VM. Running “vagrant rsync-auto” watches the local file system for changes, then runs rsync to copy changes when found.

Next I would like to do a Docker based variant. Volunteers welcome. Should it use Docker Composer with a series of small containers? Should it be a single container with everything installed? It could be interesting to have both available, allowing people to switch between both to compare.

magento-dev destroy

Deletes the environment and anything set up by the create command. For vagrant-rsync, it also destroys the Vagrant box. You would not use this command very often in real life. (I mainly use it for testing – create, destroy, create, destroy, …)

magento-dev connect <provider>

The idea is to collect whatever is required to connect to the hosting provider node, such as the host name and ssh identify file to use, or it might be ftp account details and credentials. It depends on how the hosting provider works.

magento-dev pull-code

Grabs a copy of the code currently deployed in production down to the local development environment. If you don’t use source code management, you may pull-code, make changes, push-code. If you use source code management, you may just pull-code once and commit it, instead using git and push-code per deployment.

magento-dev push-code

Puts production site into maintenance mode, pushes code to the production host, runs all required deployment scripts, does database schema upgrades, then takes site out of maintenance mode. The exact path names may be different for different hosting providers.

Where this may get trickier is if there are different versions of the hosting environment. The script may need to “sense” the version of the deployment environment (e.g. if the directory structure changes). This is a real worry for me personally – how to test code over the long term for older environments? It may not be possible to recreate an old environment (even though they are still in use).

magento-dev disconnect

Forgets the connection details saved by the connect command. Again, I mainly find it useful for testing.

Future Commands

I was wondering if there should be commands to pull/push production data, at least for testing purposes. There is some risk in this, so maybe only pull should be supported. But normally the production site is the master for product/catalog data, so for testing I could imagine it would be useful to download a copy at times – at least for those users who have not signed up for Magento Cloud! (Magento Cloud has tools to clone environments with a button click.)

Conclusions

This post is to advertise the project on GitHub. It is new, but the vagrant-rsync environment works. A Magento Cloud provider is also available which just displays instructions to the screen.

Feedback always welcome, although the code is with no guarantee of support. Types of feedback that would be useful:

• “I tried using vagrant-rsync but I found the files in ‘scripts’ directory were too annoying – they collided with other files I wanted in my project.”
• “I wrote a provider for Docker on Mac OS – would you like me to create a pull request?” (YES!)
• “When I tried pushing the code to GoDaddy, it was too hard to avoid pushing the Vagrantfile as well. It would be easier if the directory structure was changed to keep files more separate.”
• “The vagrant-rsync environment should create.gitignore and .svnignore files as well as the scripts.”
• “I created a better base box with NodeJS preinstalled, newer PHP release, etc.”
• “I think it’s important to support more than one development environment, but not too many.”
• “I think it’s important to allow solution partners to define their own deployment processes easily and plug them in.” (They can always have separate scripts, so I am not sure why this would be a requirement.)
• “I think this should be merged with the Magento CLI, so the version number of the environment and tools required is more easily kept in sync with the version of Magento.”

Feel free to leave comments here or raise issues on the GitHub repository directly.