The recent Magento 2.1.2 patch did not strictly follow the semantic versioning rules for patches. In this case we made a judgment call that the benefit to customers based on feedback we received (especially as we approached holiday peak) outweighed the negatives. The change was new service contracts were added to formalize the API to existing internal functionality. Existing client code was not be impacted (they don’t have to use the new APIs), but true semantic versioning would defer making these APIs available to the next minor release (2.2).

I thought this blog post was a good chance to talk more about what the versioning rules are and explain why Magento knowingly decided to break the rules in this case.

Semantic Versioning

Semantic Versioning (SemVer) describes a way to use version numbers in a consistent and structured way. For example, when you see a version number of 100.1.0 for a Magento module, you know it means major version 100, minor version 1, patch level 0 (that is, not patched yet). Cross module dependencies list compatible version number ranges for that dependency, ensuring you don’t accidentally mix incompatible versions of modules.

A key purpose behind semantic versioning is to allow a patch to be released that can be included into projects with confidence of not breaking other code. For example, if an API has a method removed, that cannot be included in a patch as there may be other code calling that method. Removing the method would break the other code.

In SemVer, patches are not permitted to have a public API change. This allows cross module dependencies to specify a compatible version number ranges of “100.1.*” (meaning any patch level of version 100.1) or even “100.*” (meaning any minor or patch level of major version 100 of a module).

Product vs Module Version Numbers

Product version numbers (such as 2.1.2 for Community Edition) do not follow the SemVer rules, at least for the major and minor version number. Product version numbers are marketing driven. In Magento, SemVer is used at the module level where version numbers start from 100 (to avoid confusion with product version numbers). All cross module dependencies specify module version number ranges, not product version numbers. This is why it is not a problem for product version numbers to not follow SemVer.

In addition, product patches currently lock the major and minor version numbers of all modules in the product. That is, in patch 2.1.2 only the patch level (last digit) of module version numbers is permitted to change. This is the cause of the problem expanded upon below. This rule is going to be reviewed more closely, but for now only module patches can be released in product patches (not new major or minor versions of modules).

What Rule are we Breaking?

Formally, SemVer states that the minor version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API. (See http://semver.org/#spec-item-7.) Why?

Put simply, SemVer says not to introduce new functionality (public APIs) into a patch. Patches are for bug fixes. An example of the complexity that can result from functionality being added to patches is dependency ranges need to specify the minimum patch level that functionality was introduced, rather than the preferred “any patch level of minor version X.Y”. SemVer aims to simplify dependency management by having simple rules.

So What Exactly Changed?

Patch 2.1.2 introduced a new interface to define a standard way to access existing functionality, specifically for capturing payments and shipping an order. (Other interfaces may come in the future for credit memos and emailing a paid invoice.) These were needed to support integrations with 3^rd party shipping, ERP, and other solutions. Two new interfaces were exposed as web APIs via new service contracts in the Sales module. Because they have been created as new interfaces this won’t impact any existing customizations or extensions that call the current sales APIs.

The implication is sites or extension developers that would like to use this new functionality will need to update their code to utilize the new interface and specify the minimum patch level. For example, the sales module version number from 2.1.2 or later (not 2.1.0 or 2.1.1). To be precise, it should specify 100.1.2 or later as can be seen on GitHub for 2.1.2 compared to 100.1.1 from the 2.1.1 patch.

Change Details

The introduced APIs are as follows:

• salesShipOrderV1 interface exposed as a POST web api /V1/order/{orderId}/ship
  • You can post with an empty post body and ship the entire order
  • You can do a partial shipment by specifying the order line item(s) that you’re shipping in the post body
  • If you’ve already done a partial shipment and call again with an empty post body Magento will ship the rest of the order
• salesInvoiceOrderV1 interface exposed as a POST web api /V1/order/{orderId}/invoice
  • You can post with an empty post body and capture payment for the entire order
  • You can do a partial invoice by specifying the order line item(s) in the post body
  • If you’ve already partially invoiced an order and call again with an empty post body Magento will capture the rest of the order

Why Not Increment the Module Minor Number?

So since there are module version numbers already in Magento 2 (the numbers starting from 100 onwards), why not just increment the minor number for the sales module? Then reference that from the CE/EE patch?

There is a really deep and hard to explain reason why this is a pain in practice for Magento that I will skip for now. Instead I describe the problem it creates for extension developers. If the minor version number is incremented, any extensions that depend on the current minor version number (and excludes the next minor version) may break the next time you run composer. To release a new minor version of a module may require all extensions dependent on sales entities to update their composer dependencies as well. The coordination effort here is significant. That is, it’s not a technical reason why Magento is taking this path – it is a coordination effort.

So a part of the longer term solution is around improving practices (and clarity of those practices) by both Magento and extension developers in terms of version number management of modules. (Magento has a responsibility to clearly communicate the rules to extension developers, and then help them follow the rules.) For example, if extensions who only call the Sales APIs consistently used dependency of 100.* (minor number not specified), it would not have been a problem releasing a new minor version number. Unfortunately, our investigation shows this is not the case today – different extensions use different forms of dependencies without much consistency.

Conclusion

It is understood that adding new functionality (exposed as web APIs) in a patch release is not in strict keeping with semantic versioning.  However, we believe it’s the right thing to do to address the needs of our customers and partners in the most expedient way possible. We have received compelling feedback that the missing functionality is worth adding to 2.1 and not waiting for 2.2 to be released. The change should have no impact on clients of the Sales API as they are completely new interfaces. But it was an interesting example of the trickiness of version management.

Researching into what existing extensions are specifying in terms of dependencies also revealed that Magento needs to put more investment into educating extension developers on how to do Composer based dependencies correctly. Increasing the minor version number, following SemVer strictly, may have been possible in the 2.1.2 patch if this was the case. (Having Magento Marketplace where Magento hosts the code of 3rd party extensions is proving very useful for performing this sort of research – Magento has better insight into the impact of changes across all extension developers.)

Side note: I originally posted a draft of this post publicly by mistake, which some people saw. This is now public and the final form. Sorry for any confusion!