Sitecore Cortex and ML: Part 2 - How to Add Custom Events, Facets, and Models

For our demo scenario, we create a custom outcome event “Demo Purchase” that will be triggered when a customer buys a product. We will store information about customer purchase within this event. In the next posts, we will use our outcome model to generate the initial data model for Cortex (projection model).

Product price indicates a major value for all our processes. We need to mark our outcome to have ‘Monetary Value Applicable’:

Next, we create the corresponding class with some additional properties:

• CustomerId - stores contact identifier
• InvoiceId - stores purchase identifier
• Quantity - stores count of products that customer purchased

public class PurchaseOutcome: Outcome
{
    public int Quantity { get; set; }
    public int InvoiceId { get; set; }
    public int CustomerId { get; set; }

    public PurchaseOutcome(Guid definitionId, DateTime timestamp, string currencyCode, decimal monetaryValue, int invoiceId, int quantity, int customerId) 
        : base(definitionId, timestamp, currencyCode, monetaryValue)
    {
        this.Quantity = quantity;
        this.CustomerId = customerId;
        this.InvoiceId = invoiceId;
    }

    // ID of "Demo Purchase" item
    public static Guid PurchaseEventDefinitionId { get; } = new Guid("F841E7C8-A12D-4376-94CD-296A37EDABCC");
}

To make our PurchaseOutcome visible for xConnect and xConnect jobs, we need to build a custom model where we register our new event type PurchaseOutcome.

public static class XdbPurchaseModel 
{
    public static XdbModel Model { get; } = BuildModel();

    private static XdbModel BuildModel()
    {
        XdbModelBuilder modelBuilder = new XdbModelBuilder("PurchaseOutcome", new XdbModelVersion(1, 0));

        modelBuilder.ReferenceModel(Sitecore.XConnect.Collection.Model.CollectionModel.Model);
        modelBuilder.DefineEventType<PurchaseOutcome>(false);
        return modelBuilder.BuildModel();
    }
}

Run the following code to generate json file:

var fileName = XdbPurchaseModel.Model.FullName + ".json";
            var json = XdbModelWriter.Serialize(model);
            System.IO.File.WriteAllText(fileName, json);

Copy generated json file to Model folder of the xConnect and Model folder of xConnect jobs:

1. xconnect_instance\App_Data\Models\
2. xconnect_instance\App_Data\jobs\continuous\ProcessingEngine\App_Data\Models\
3. xconnect_instance\App_Data\jobs\continuous\AutomationEngine\App_Data\Models\
4. xconnect_instance\App_Data\jobs\continuous\IndexWorker\App_data\Models\

Important: We also need to register XdbPurchaseModel in xConnect client configuration of the Processing Engine job. If we don`t do it - the projection will not work. To do it we need to patch “xconnect_instance\App_Data\jobs\continuous\ProcessingEngine\App_Data\Config\Sitecore\XConnect\sc.XConnect.Client.xml” config:

<Settings>
    <Sitecore>
        <XConnect.Client>
            <Services>
                <Client.Configuration>
                    <Type>Sitecore.XConnect.Client.Services.Configuration.XConnectModelConfigurator, Sitecore.XConnect.Client.Services.Configuration</Type>
                   <As>Sitecore.XConnect.Client.Services.Configuration.IXConnectModelConfigurator, Sitecore.XConnect.Client.Services.Configuration</As>
                    <LifeTime>Transient</LifeTime>
                    <Options>
                        <!-- The xDB models to use with the xConnect client configuration. -->
                        <Models>
                            <DefaultModel>
                                <TypeName>Sitecore.XConnect.Collection.Model.CollectionModel, Sitecore.XConnect.Collection.Model</TypeName>
                                <PropertyName>Model</PropertyName>
                            </DefaultModel>
		    <CustomModel>
<TypeName>Demo.Foundation.ProcessingEngine.Models.XdbPurchaseModel, Demo.Foundation.ProcessingEngine</TypeName>
                                <PropertyName>Model</PropertyName>
                            </CustomModel>
                        </Models>
                    </Options>
                </Client.Configuration>
            </Services>
        </XConnect.Client>
    </Sitecore>
</Settings>

To make our model “visible” for Automation Engine we also need to add it to Automation Engine xConnect client configuration:

“xconnect_instance\App_Data\jobs\continuous\AutomationEngine\App_Data\Config\sitecore\sc.XdbPurchase.Model.xml” (xml file name should be in format sc.CustomName.Model.xml):

   <Settings>
   <Sitecore>
    <XConnect>
      <Services>
        <XConnect.Client.Configuration>
          <Options>
            <Models>
              <XdbPurchaseModel>
                <TypeName>Demo.Foundation.ProcessingEngine.Models.XdbPurchaseModel, Demo.Foundation.ProcessingEngine</TypeName>
              </XdbPurchaseModel>
            </Models>
          </Options>
        </XConnect.Client.Configuration>
      </Services>
    </XConnect>
  </Sitecore>
</Settings>

Finally, If we want to use our custom PurchaseOutcome model within the xdb tracker, we need to create an event conversion pipeline processor and add our model configuration to the xConnect schema of our website instance.

public class ConvertPurchaseOutcome : ConvertToXConnectEventProcessorBase<OutcomeData>
{
        protected override Event ConvertToEvent(OutcomeData entity)
        {
            var quantity = int.Parse(entity.CustomValues["Quantity"] as string);
            var invoiceId = int.Parse(entity.CustomValues["InvoiceId"] as string);
            var contactId = int.Parse(entity.CustomValues["CustomerId"] as string);

            var purchase = new PurchaseOutcome(PurchaseOutcome.PurchaseEventDefinitionId, entity.Timestamp,  entity.CurrencyCode, entity.MonetaryValue,invoiceId,quantity, contactId);

            return purchase;
        }

        protected override bool CanProcess(Sitecore.Analytics.Model.Entity entity)
        {
            if (entity is OutcomeData)
            {
                OutcomeData outcomeData = (OutcomeData)entity;
                return (outcomeData.OutcomeDefinitionId == PurchaseOutcome.PurchaseEventDefinitionId);
            }

            return false;
        }
}

Add new config “Foundation.ProcessingEngine.config” to website instance:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <pipelines>
            <convertToXConnectEvent>
                <processor patch:before="processor[@type='Sitecore.Analytics.XConnect.DataAccess.Pipelines.ConvertToXConnectEventPipeline.ConvertOutcomeDataToOutcome, Sitecore.Analytics.XConnect']" type="Demo.Foundation.ProcessingEngine.Processors.ConvertPurchaseOutcome, Demo.Foundation.ProcessingEngine"/>
            </convertToXConnectEvent>
        </pipelines>

      <xconnect>
        <runtime type="Sitecore.XConnect.Client.Configuration.RuntimeModelConfiguration,Sitecore.XConnect.Client.Configuration">
          <schemas hint="list:AddModelConfiguration">
            <!-- value of 'name' property must be unique -->
            <schema name="purchaseoutcomemodel" type="Sitecore.XConnect.Client.Configuration.StaticModelConfiguration,Sitecore.XConnect.Client.Configuration" patch:after="schema[@name='collectionmodel']">
              <param desc="modeltype">Demo.Foundation.ProcessingEngine.Models.XdbPurchaseModel, Demo.Foundation.ProcessingEngine</param>
            </schema>
          </schemas>
        </runtime>
      </xconnect>
    </sitecore>
</configuration>

Now we can trigger our custom event in code when user makes a purchase:

var customer = Tracker.Current.Session.Contact.Identifiers.First(x => x.Source == "demo");
       
var ev = Tracker.MarketingDefinitions.Outcomes [PurchaseOutcome.PurchaseEventDefinitionId];
var outcomeData = new Sitecore.Analytics.Data.OutcomeData(ev, "USD", 100);
outcomeData.CustomValues.Add("Quantity", "5");
outcomeData.CustomValues.Add("CustomerId", customer.Identifier);     outcomeData.CustomValues.Add("InvoiceId", "10001");
Tracker.Current.CurrentPage.RegisterOutcome(outcomeData);

Or we can also register our custom outcome programmatically. (Note: if you want to register outcome with webvisit event, pass addWebVisit=true parameter and replace channel and page Ids with necessary values):

public async Task<bool> Add(Customer purchase, bool addWebVisit = false)
    {
        using (XConnectClient client = SitecoreXConnectClientConfiguration.GetClient())
        {
            try
            {

                IdentifiedContactReference reference = new IdentifiedContactReference(IdentificationSource, purchase.CustomerId.ToString());
                var customer = await client.GetAsync(
                    reference,
                    new ContactExpandOptions(
                        PersonalInformation.DefaultFacetKey,
                        EmailAddressList.DefaultFacetKey,
                        ContactBehaviorProfile.DefaultFacetKey
                    )
                    {
                        Interactions = new RelatedInteractionsExpandOptions
                        {
                            StartDateTime = DateTime.MinValue,
                            EndDateTime = DateTime.MaxValue,
                            Limit = 100
                        }
                    }
                );

                if (customer == null)
                {
                    var email = "demo" + Guid.NewGuid().ToString("N") + "@gmail.com";

                    customer = new Contact(new ContactIdentifier(IdentificationSource, purchase.CustomerId.ToString(), ContactIdentifierType.Known));

                    var preferredEmail = new EmailAddress(email, true);
                    var emails = new EmailAddressList(preferredEmail, "Work");

                    client.AddContact(customer);
                    client.SetEmails(customer, emails);

                    var identifierEmail = new ContactIdentifier(IdentificationSourceEmail, email, ContactIdentifierType.Known);
                    
                    client.AddContactIdentifier(customer, identifierEmail);
                    client.Submit();
                }

                // existing 'Other 3rd party' channelId
                var channel = Guid.Parse("DF9900DE-61DD-47BF-9628-058E78EF05C6");
                var interaction = new Interaction(customer, InteractionInitiator.Brand, channel, "demo app");

                if (addWebVisit)
                {
                    //Add Device profile
                    DeviceProfile newDeviceProfile = new DeviceProfile(Guid.NewGuid()) { LastKnownContact = customer };
                    client.AddDeviceProfile(newDeviceProfile);
                    interaction.DeviceProfile = newDeviceProfile;

                    //Add fake Ip info
                    IpInfo fakeIpInfo = new IpInfo("127.0.0.1") { BusinessName = "Home" };
                    client.SetFacet(interaction, IpInfo.DefaultFacetKey, fakeIpInfo);

                    // Add fake webvisit
                    // Create a new web visit facet model
                    var webVisitFacet = new WebVisit
                    {
                        Browser =
                            new BrowserData
                            {
                                BrowserMajorName = "Chrome",
                                BrowserMinorName = "Desktop",
                                BrowserVersion = "22.0"
                            },
                        Language = "en",
                        OperatingSystem =
                            new OperatingSystemData { Name = "Windows", MajorVersion = "10", MinorVersion = "4" },
                        Referrer = "www.google.com",
                        Screen = new ScreenData { ScreenHeight = 1080, ScreenWidth = 685 },
                        SearchKeywords = "sitecore",
                        SiteName = "website"
                    };

                    // Populate data about the web visit
                    // HomePage ID
                    var itemId = Guid.Parse("110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9");
                    var itemVersion = 1;

                    // First page view
                    var datetime = purchase.Invoices.FirstOrDefault() == null
                        ? DateTime.Now
                        : purchase.Invoices.First().TimeStamp.ToUniversalTime();
                    PageViewEvent pageView = new PageViewEvent(datetime,
                        itemId, itemVersion, "en")
                    {
                        ItemLanguage = "en",
                        Duration = new TimeSpan(3000),
                        Url = "/home"
                    };
                    // client.SetFacet(interaction, WebVisit.DefaultFacetKey, webVisitFacet);
                    interaction.Events.Add(pageView);
                    client.SetWebVisit(interaction, webVisitFacet);
                }



                foreach (var invoice in purchase.Invoices)
                {
                    var outcome = new PurchaseOutcome(PurchaseOutcome.PurchaseEventDefinitionId, invoice.TimeStamp, invoice.Currency, invoice.Price, invoice.Number, invoice.Quantity, purchase.CustomerId);
                    interaction.Events.Add(outcome);
                }

                client.AddInteraction(interaction);

                await client.SubmitAsync();

                return true;
            }
            catch (XdbExecutionException ex)
            {
                Log.Error(ex.Message, ex, this);
                return false;
            }
        }
    }

Last but not least, we need to create our custom contact facet to store calculated RFM values and predicted cluster.

 [Serializable]
    [FacetKey(DefaultFacetKey)]
    public class RfmContactFacet : Facet
    {
        public const string DefaultFacetKey = "RfmContactFacet";

        public RfmContactFacet()
        {

        }
        public int R { get; set; }
        public int F { get; set; }
        public int M { get; set; }
        public double Recency { get; set; }
        public int Frequency { get; set; }
        public double Monetary { get; set; }

        public int Cluster { get; set; }
    }

To make our RfmContactFacet visible for the xConnect and xconnect jobs, again we need to build custom model where we register our new facet.

public static class XdbPurchaseContactModel
    {
        public static XdbModel Model { get; } = BuildModel();

        private static XdbModel BuildModel()
        {
            XdbModelBuilder modelBuilder = new XdbModelBuilder("ContactModel", new XdbModelVersion(1, 0));

            modelBuilder.ReferenceModel(Sitecore.XConnect.Collection.Model.CollectionModel.Model);
            modelBuilder.DefineFacet<Contact, RfmContactFacet>(RfmContactFacet.DefaultFacetKey);

            return modelBuilder.BuildModel();
        }
    }

Generate json file:

var fileName = XdbPurchaseContactModel.Model.FullName + ".json";
var json = XdbModelWriter.Serialize(model);
System.IO.File.WriteAllText(fileName, json);

Copy this generated json file to Model folder of xConnect and Model folder of xConnect jobs:

1. xconnect_instance\App_Data\Models\
2. xconnect_instance\App_Data\jobs\continuous\ProcessingEngine\App_Data\Models\
3. xconnect_instance\App_Data\jobs\continuous\AutomationEngine\App_Data\Models\
4. xconnect_instance\App_Data\jobs\continuous\IndexWorker\App_data\Models\

Here are some links that will help you understand custom outcome programming better:

https://doc.sitecore.com/developers/90/sitecore-experience-platform/en/create-a-custom-outcome-model.html
https://doc.sitecore.com/developers/90/sitecore-experience-platform/en/triggering-custom-events.html
https://doc.sitecore.com/developers/90/sitecore-experience-platform/en/convert-an-outcome.html

Table of contents Dive into Sitecore Cortex and Machine Learning - Introduction

Read next Part 3 - Processing engine Projection models and Datasets

Do you need help with your Sitecore project?

VIEW SITECORE SERVICES