Introducing the Sitecore Commerce 9 Connector for Microsoft D365

At Sitecore Symposium 2019 this year, I collaborated with my coworker and friend Vsevolod Kolonistov to present the new Sitecore Commerce 9 connector for Microsoft D365 Retail.

We’d gained a lot of experience working with the prior version of the connector, so when it came time to modernize it to use the new Sitecore Commerce 9 infrastructure, we had a lot of advice and knowhow to share with the Sitecore Commerce team. We also wanted to share it with the rest of the Sitecore community, and where better to do that than at Symposium?

Sitecore Commerce and Microsoft D365 Retail are an interesting combination. While Sitecore Commerce alone is more than capable of powering B2C ecommerce websites right out of the box, if you also need to worry about physical points-of-sale or call center operations, you’ll need to pair it with another enterprise-grade tool. Which is where Microsoft D365 Retail comes in.

The integration is an interesting mix:

  • Batch operations for syncing catalog information and product media¬†
  • Real-time operations for cart, checkout, and customer accounts

But I won’t go into all the details here — you can get the slides below.

Yes, Symposium was such a whirlwind that I’m only now posting the slides from the breakout session!

Presenting the Sitecore-D365 connector at Symposium 2019
Dean Thrasher (left) and Vsevolod Kolonistov (right) presenting the new Sitecore Commerce 9 connector for Microsoft D365 Retail

One thing I neglected to mention in the slides themselves, though it was covered in the Q&A session, is that the D365 Connector isn’t available from the Sitecore developer site directly. For now, if you’re interested, you’ll need to contact your Sitecore representative directly.

I’m curious to see how many other companies pursue this sort of integration. D365 Retail — a specialization of D365 for Finance and Operations — is still evolving rapidly. Microsoft wants it to be a enterprise-grade SaaS ERP, but you can still see a lot of the old Dynamics AX client-server architecture peeking through in places. I expect it to improve with time, though, so perhaps we’ll see more of this Sitecore-D365 combination in the future. If so, we’ve got the connector ready for it!

Sitecore Symposium 2019 Customer Showcase

Every year at Sitecore Symposium, Sitecore features customer success stories. This year, I had the honor to co-present the relaunch of the Lowe’s Canada website with Tanbir Grover, VP of Digital and Omnichannel at Lowe’s Companies Canada.

Sitecore Symposium 2019 customer showcase
On the big stage at Sitecore Symposium 2019

In August 2019, after nearly 3 years of work (if you include the the first proof-of-concept in 2016), the new website went live across Canada. The results so far have been very promising, especially for mobile ecommerce. Tanbir was able to share some impressive statistics. More important than the numbers, though, is the improvements in flexibility and agility.¬†The retail sector has been reinventing itself, and now the digital team at Lowe’s Canada can respond to changing trends and innovate in key areas such as content marketing, personalization, and search.

I wish we’d been able to put the whole team on stage — from Lowe’s Canada and EPAM. This project would not have succeeded without a lot of very smart people from around the world solving difficult commerce problems at scale. I’ve very proud to have been a part of that team.

SCpbMD Catalog Sync Explained

I’ve been meaning to post this diagram for a while. I’ve used this to explain the Sitecore Commerce catalog data sync operation to at least three different clients since I drafted it this summer. And although I created it for the Microsoft Dynamics 365 version of the connector, it’s similar for Dynamics AX and other PIM systems as well.

SCpbMD_DataSync_2017-11-27.png

In the D365 box, you have the UI application, which admins can use to publish catalog data once it has been validated. There are a lot of elements that must be configured an working correctly to have a valid catalog, but a few of the key pieces are:

  • An online navigation hierarchy for your online channel
  • An assortment for your online channel containing released products
  • A catalog associated with the online channel
  • Products assigned to nodes on the online navigation hierarchy
  • Product attributes defined and attached to nodes on your online navigation hierarchy

Once the catalog is published, it’s ready to go as far as the “headquarters” database is concerned. But in Dynamics AX / D365, it also has to be distributed — sent to the online channel database using distribution jobs.

Once the catalog is in the online channel database, the D365 Retail Server can read from it. External applications can read catalog data from the online channel database using the Retail Server APIs, a curious mix of web services that aren’t quite WCF and aren’t quite REST.

This is where the Sitecore part of the picture comes into play. Sitecore provides a sample console application that uses Sitecore’s Data Exchange Framework to fetch data from the Dynamics Retail server. It transforms it into an XML file that can then be imported into Sitecore’s Commerce Server. (Sitecore 9 also uses this catalog.xml file format, though the old commerce server components are no longer used.)

This places the product and category definitions and data into the product catalog database. This product catalog database acts as an “edge cache” that keeps just the products the site will use close to the infrastructure of the website itself. It provides some redundancy in case that communications problems occur between Sitecore and D365.

The last step in the process is the catalog data provider. Sitecore XP uses a data provider to access the product catalog database, creating virtual Sitecore items that appear within the Sitecore UI. Product and category data are not stored as “real Sitecore items” in the sense that they live in the standard Sitecore master or web databases.

Watch the arrows!

Note the color and direction of the arrows in the diagram above. The orange arrows are the ones controlled by the Sitecore console app and Data Exchange Framework. The arrows in blue are either part of standard D365 functionality or belong to extensions to the Sitecore platform. The orange arrows could have been labeled “Extract, Transform, and Load” because that’s exactly the operations performed by the catalog sync. (If I ever redraw the diagram, I might update the labels to say just that!)

The direction of the arrows are important, too. Catalog information must be sent from AX HQ to the channel database by those batch distribution jobs. If those jobs aren’t running, then no updates occur in the channel DB, and no updates will be returned in by the Retail Server API.

Once the data is available at the Retail Server, it’s up to the Sitecore catalog sync process to fetch the latest data from the Retail Server. This can be run manually or as a scheduled job, but note that there are no notifications here — D365 doesn’t push the data to Sitecore, Sitecore pulls the data when it needs to.

A sequence of batches

This is definitely not a real-time process. As you might imaging from the number of batches, pushes, and pulls shown in the diagram, it can take a significant amount of time to move an update — like a adding a new product to the catalog or setting up a new product attribute for a category — from D365 HQ into Sitecore XP. If every batch job involved executed on a 15 minute timer, it could take 45-60 minutes for that product to appear on the site. The interval could be longer depending on the size of the catalog and the number of changes made.

There’s more than one way to do it

Although Sitecore provides the catalog sync code as part of its commerce connectors, it’s really just an example or starter kit for us to use. In practice, you’ll need to modify the logic used to generate the catalog.xml file to import into Sitecore. You may also need to move the data sync process to other servers for scalability or performance reasons. Or you could replace Sitecore’s Data Exchange Framework with another ETL framework or a business process orchestration suite like BizTalk.

The connector is just a starting point for implementation, and hopefully the diagram and my explanation of it makes a good starting point for discussion with your team or client about how the process of syncing catalogs might work.

Sitecore Commerce Catalogs at Scale

Last week, I gave a presentation to the DC Sitecore User Group on Sitecore Commerce Catalogs. It was a small crowd due to some thunderstorms in the area, and I had a tough act to follow. Phil Wickland, Sitecore MVP and author of several books on Sitecore, gave a talk on Personalization for Impact, which is worth seeing.

My talk was about how and why Sitecore imports catalog data from a PIM, using the Sitecore Commerce and Microsoft D365 integration as an example.

Here’s a link to the video:

The audio is a bit hard to hear at times, but I’ve posted my slides to slideshare here: