This blog describes a proposal of how you can use Gulp with Magento 2 today. It requires extension developers to do CSS in a particular way which is not currently recommended/documented. Feedback welcome to help determine if the approach is appropriate to turn into official policy.

The Scope

There are bigger questions – why use Less instead of Sass, why not use Angular, why not use Laravel, why not use Flash? (Okay, I made the last one up!) I am not going to talk about long term strategy in this post – I am going to focus on getting a more speedy development approach going today. That is, this post is purposely tactical, not talking big picture.

The Problem

There are several problems including:

• The default Magento build chain for CSS files is much slower than what most frontend developers are used to. And it does not support ‘watching’ for file changes (“live reload”).
• Magento added support for Grunt, but there is a little “magic” around it.
• Gulp is faster than Grunt, probably because it does more processing in memory. See also https://github.com/poddubny/magento2-gulp which reported 2 to 3 times faster.
• There are newer build technologies coming every day. As a few have noted (yes, I do listen!), building anything into Magento is risky.

The Cause

The problem is caused by the fact that some of the core Magento Less files use a non-standard form of Less preprocessing. This means you cannot use standard tools and take advantage of the optimizations they have achieved. You have to use the Magento tools with a Less preprocessor in PHP. This made the default experience simpler, at the price of slowing down experts.

One example is the //@magento_import directive which searches through all modules for matching files and imports them. This is useful for Magento in that it allows CSS to be put in the same modules as the HTML that needs it, which allows us to make modules optional. However it causes problems because standard tools such as Grunt and Gulp do not understand it. If you look in app/design/frontend/Magento/blank/web/css/styles-l.less you will find the following lines:

 //@magento_import 'source/_module.less'; // Theme modules
 //@magento_import 'source/_widgets.less'; // Theme widgets
 ...
 //@magento_import 'source/_extend.less';

What these statements do is allow every module to declare a _module.less, _widgets.less, or _extend.less file to contribute into the base styles-l and styles-m CSS files (one for large displays, the other for mobile). While useful, the lack of compatibility with standard Less tools is creating more pain than gain for “serious” front end developers.

Another problem is Magento uses a “file fallback” scheme, which is also not standard Less technology. For example, you can have a single file that is automatically shared across multiple locales, with overrides per locale when needed. This has advantages such as the code will work reasonably (using defaults) even for unplanned locales. There are multiple uses of the fallback scheme which gets into a long discussion which I am not going to get into here – the point is it is not supported by Less.

The Wall

Meme courtesy Mark Shust on twitter

The solution I propose here is based on putting a wall between the internal Magento code and external tools such as Gulp. The goal is to use tools such as Gulp with no special Magento magic in sight. Gulp users should not look over the wall at what is on the other side. Long term we can look at breaking down the wall and getting everything working better, but for now let’s get people going with what we have at hand.

Let’s consider a frontend developer working on a single theme for one locale. So there is a specific area/theme/locale triple that the frontend developer is working on. (If they are working on multiple, just repeat the following per combination.)

The command magento dev:source-theme:deploy (used also by Grunt integration) outputs a directory tree of Less files for the specified area/theme/locale triple with all //@magento_import commands expanded and all “file fallback” occurrences resolved. So the concept of the wall is to tell front end developers “do not look at the core Magento source files, but rather look at the output of this command”. (This is the wall.) Standard Less tools work on the output directory tree. For example, the “frontend” area, “Magento/luma” theme, “en_US” locale is one such directory tree created. This is created in the directory pub/static/frontend/Magento/luma/en_US.

(Why does it go in pub/static? It helps with client side Less preprocessing. If Magento drops that functionality this directory could be put under say var.)

How the command works is it creates symlinks for most files back to the master files, meaning if you do edits in either the generated directory tree or on the original file, the original file will be updated. (Gulp will spot changes to the original file when watching symlinks, so this works nicely.)

In addition, all the path names in the output directory are relative paths – all “fall back” file references are resolved.

So when to run this command? As a general rule, if you add or remove (or rename) a file, run the command again. Otherwise don’t. When messing around with CSS tweaking, most of the time you are changing the same file again and again so this is not too painful.

The Catch

The catch is this works best if extensions do not take advantage of the //@magento_import command. That is, if they need additional styles defined, put them in a separate CSS file and include that file in the module. (I think it still works if they do leverage //@magento_import, but it is more “natural” if they use a separate file.)

How to include the additional CSS file? In the appropriate layout file, add a <css> element instead the <head> element of the layout file. It is important to include the module name via the “::” prefix to the path. For example, <css src=”Magento_Customer::css/myfile.css”/>. That will cause the page to reference that CSS file and include the CSS file in themes under Magento_Customer avoiding collisions with other modules.

If you want the efficiency from loading fewer CSS files (rather than lots of little ones per module), you can either use the Magento “merge CSS files” capability in production (in dev it messes with Gulp again), or write your own Less file to import all the individual files and use the layout file <head>/<remove> instruction to remove the CSS references. (E.g. see discussion in http://magento.stackexchange.com/questions/99239/remove-pub-media-styles-css-from-theme-in-magento-2-0/99255#99255.)

But what if the file wants to use the Less mixins in the Magento UI Library? The answer turns out to be fairly simple. Just @import the files source/lib/_lib.less (the entrypoint to the Magento UI Library) and source/_theme.less (the override file for themes to use to globally change mixin variables). (If you look in source/lib you will notice each mixin is in its own file.)

But what if an extension already uses _module.less or _extend.less? Shhh, don’t tell Ben Marks, but you EDIT THE CORE! But it’s okay, it’s not the Magento core… Well, alright, it’s not a great solution but it works for now. (This is not strictly required, so maybe try in experiments but don’t do in real production.) Just rename the file to something else and @import that. If the proposal in this blog post works out, we will recommend extension developers follow the new approach to fix this problem for the future. But I don’t want to make that general recommendation until I see a bit of “hey this works well” type feedback from the community. Any help appreciated!

The Solution

So how does this all fit together?

• Run “magento dev:source-theme:deploy” whenever you add or remove a file.
• Set up Gulp or similar on the pub/static/frontend/… directory for your theme where the Less files are written to. That is, watch that directory using something like **/*.less, and write the CSS file to that directory.
• For new CSS files, don’t use _extend.less or _module.less, instead load the CSS file using <head>/<css>. That file can still use mixins using @import of the source/lib/_lib.less and source/_theme.less files (in that order).

That’s it!

The Community

Well, I put a short paragraph off somewhere a few days ago as a precursor to this post. So of course a few community projects sprung up already demonstrating Gulp working, faster than I could finish writing this blog post. (Please leave any others in the comments section below.)

• https://github.com/poddubny/magento2-gulp
• https://github.com/SnowdogApps/magento2-frontools

The Conclusion

There is a way to use Gulp with Magento 2 now. My request to the wonderful (frontend developer) community is to give it a go and let me know how well it works. We can then weave that back into the core product in a backwards compatible way to move forwards. If extensions do not use the _module.less, _email.less, or _extends.less files but instead use separate CSS files for additional CSS they need, then this should all just work today.

We do have some ideas on how to eliminate the need for “the wall”, but they would involve core product changes. The above should work right now, if you use the tools appropriately. If it gets a thumbs up, we can take the next step from there.