Magento 2 Digital Commerce (the traditional Magento web store) and Commerce Order Management (a SaaS offering from Magento) communicate using JSON over AMQP. This post describes some initial messages we support and our plans to open up the protocol to the wider partner and potentially extension developer ecosystem.

Digital Commerce? Commerce Order Management? Huh?

Magento Digital Commerce (MDC) is the name now given to the traditional Magento web store. It has been given this name due to the addition of new products to the Magento product line. Magento Commerce Order Management (MCOM) is a SaaS based order management system providing more advanced functionality such as inventory management across multiple warehouses, ship from store, advanced payments processing features, and more. There is also Magento Marketplace, the “app store” for Magento extensions (going live shortly).

But old habits die hard. Even our internal staff still say “Magento” meaning “Magento Digital Commerce”. This change is going to take a while to settle in.

Service Contracts and AMQP in MDC

MDC supports “service contracts” which are PHP interfaces that follow certain rules, making it easier to bind various transport protocols (such REST, SOAP, and in Enterprise Edition AMQP) to implementations of the PHP interfaces via configuration files (no source code change required). The Enterprise Edition of MDC 2.0 had elementary AMQP support (RabbitMQ in particular), whereas 2.1 has fleshed out more such functionality for both sending and receiving requests. Still more functionality in this area is planned in future releases.

One fundamental goal of this approach is to separate business logic (as written by extension developers) from transport bindings to REST, SOAP, and AMQP (as configured by systems integrators). This allows system integrators to set up a particular project for a merchant without having to change the source code provided by an extension developer.

For example, when writing code to react to a request, a developer defines a PHP interface and provides a class implementing the interface. There is no mention of REST, SOAP, or AMQP in the code the developer writes. They just implement an interface. To send a request is similar. The developer writes a PHP interface and in the calling code declares a constructor parameter using that PHP interface as the parameter type. To send a request, the code does a function call on a method of the interface.

Thus a programmer writes exactly the same code if one module is calling another locally loaded module as if an outbound request is made. (Outbound is only fully supported in EE 2.1 over AMQP for outbound requests, but outbound requests over HTTP are planned in a future release.) As the core code base moves towards all module interactions being over well-defined APIs, the more flexibility that will be available to system integrators to distribute a Magento installation across multiple servers if they desire. More importantly, the easier it will be to replace default implementations of modules with third party implementations. The Magento Share Services as described in this blog are another step in this direction.

Coming back to the present, the Magento 2 MDC/MCOM communication channel is JSON encoding of requests and responses over AMQP (RabbitMQ to be precise). AMQP has a number of benefits such as supporting a publisher/subscribing model and improved resiliency against failures. For example, if MDC receives a new order, it places it on a queue to be sent to MCOM. If the connection MDC and MCOM is temporarily unavailable, the order is queued for later processing.

Shared Service Specifications

Service contracts in MDC are implemented using PHP interfaces. An equivalent encoding in XML has been developed in order to be programming language neutral. This allows the MDC PHP interfaces to be generated from this XML specification. Internally, the MCOM binding code is generated from the same XML specification. Over time, additional tools for generating bindings for other languages such as Java, C#, Python, Ruby, Go, and so on can be developed.

It should be noted that today the XML of the specification is still in a state of flux. We are continuing to work through issues, and adjust as required. A part of the reason for this post is to make it available for comment by interested parties (most likely solution partners initially) before we lock it down.

It should also be noted that the fact that the spec is in XML is not that important. It could have been represented in JSON or similar. XML was chosen as it has the advantage of XSD’s which allow good IDEs to validate the XML documents as they are written (with hints of what elements can go next). The exact format does not matter that much. It is the messages encoded in the specification that matters.

Example Specifications

The specification of some initial messages can be currently found in the GitHub repository https://github.com/alankent/shared-services. Once more stable, this will move under the “magento” GitHub organization. This repository also includes additional documentation. The documentation of services is considered as important as the specification itself.

There are three types of interaction patterns used in the specification:

• Commands consist of requests and optional responses. (A request without a response is truly asynchronous, but some requests have a guaranteed response where the requester is expected to wait for the response before proceeding.) Commands typically change state of the system. For example, “create order” is a command (a new order will be added to the system), where the response may contain the order ID.
• Queries consist of a request/response pair. The intent of a query is to return information without making any change to the state of the target system. For example, “fetch order status” is a query. The order is not changed by making the request.
• Events consist of broadcast messages (publisher/subscriber) indicating the successful completion of some action. For example, “a new order has been created” can be sent to all interested parties, once the order has passed all validation and has been accepted by the system. (There has been some debate whether to rename “events” to “notifications” to avoid confusing with the existing MDC event mechanism.)

An example of a command is “update a product description” (“magento.catalog.update”, as found in https://github.com/alankent/shared-services/blob/master/catalog/product.xml#L12), of a query is “magento.catalog.find” (https://github.com/alankent/shared-services/blob/master/catalog/product.xml#L77), and “magento.catalog.deleted” (https://github.com/alankent/shared-services/blob/master/catalog/category.xml#L47) is an event is to inform others when a product has been deleted.

More detailed documentation can be found in the same GitHub repo as the specifications, and more details will be published over time to the same repo.

Why Make it Public?

You may be wondering why I am writing a post about the internal protocol used by MDC and MCOM to communicate with each other. The reason is simple. As the protocol is fleshed out in more areas with greater detail, this protocol and specifications may become of interest to solution providers tasked with integration of multiple technologies.

That is, instead of PHP code level integrations typical of MDC extensions, RabbitMQ (AMQP) level integrations of JSON data structures can be used with a range of programming languages.

As the breadth of coverage increases, the more it will be feasible for system integrators to replace services with alternative implementations. The purpose of this approach is around alignment with the greater breadth of commerce services, not just services needed by an online web store. This approach also directly aligns with the movement within the MDC code base towards service contracts. Yes, there is a grand plan behind all these changes.

The ultimate intent is to make it easier to build up a community of extensions around MCOM as well as MDC. As MCOM is a SaaS based closed source product, integrations cannot be at the code level but instead at the JSON over AMQP level. For example, a loyalty product may want to be informed of orders place by registered customers. With this new approach, it can be done without code level integration. That simplifies the effort of such integrations.

Conclusions

This approach is basically a standard Service Oriented Architecture, using JSON + AMQP as the recommended transport. To me, the technology is not the interesting part. It is formulating standardized documentation and definitions on how different systems integrate that is the change. This opens up other forms of integrations in the future.

For those who want to learn more, I recommend reading through the documentation at https://github.com/alankent/shared-services. And if you are coming to Imagine 2016, a Bar Camp session (Wednesday afternoon) is planned with a proof of concept PIM integration by Balance Internet.