Smiling and dialling with machine learning: how to call the right customers

Consider the following hypothetical phone call:

Telemarketer: Hello sir, we're calling you today from ABC Bank to let you know you are eligible for our Platinum Rewards credit card. Would you like to start the application process for the card over the phone?

How would you react to this call? Some of you may simply ignore the offer, and think nothing more of it. Others may think 'that call was unnecessary, I was planning on applying for the credit card anyway'. Others may be genuinely pleased to receive the call, and will apply for the credit card because their bank got in contact. The final group may be so annoyed at the cold call that they will cancel their account with the bank and never do business with them again.

The bank must consider the high cost per call (call centres are far more expensive than emails). They also need to understand the cost of irritating a large section of their customer base that will be forever put off by the marketing. Customers such as these are referred to as 'sleeping dogs' in the literature. The groups can be divided as follows:

How can a data scientist help a company call (or mail, or email) the right customers? Two approaches may be suggested.

Approach 1: Let's build a propensity model

Propensity models are exceedingly common in the analytics departments of many large organisations. In simple terms, a propensity model predicts the likelihood that a customer will purchase a product. The model is then used to generate a list for the marketing department that tells them who to call first. Sounds like what we need right? Wrong. A colleague of mine proposed an excellent analogy:

Say you are a doctor, and you have 100 patients with a disease. You only have the resources to treat 20 of them, so which 20 do you treat? A propensity model would tell you which patients were likely to get better on their own, independent of treatment. Does it make sense to give medicine to the patients who are highly likely to get well anyway? Obviously not. Instead, the doctor should treat the patients who are likely to get better because they receive medicine.

More often than not, companies are marketing to people already likely to purchase the product anyway. Sometimes it works, but fundamentally it is the wrong approach. What you want to be modelling is the probability a customer will buy a product if you market to them. This is a subtle, but critically important difference to the propensity modelling approach.

Approach 2: Uplift modelling

Despite the fact that models in the first approach are trained to predict propensity, they are almost always evaluated on a different metric: uplift. From Wikipedia:

The uplift of a marketing campaign is usually defined as the difference in response rate between a treated group and a randomised control group. This allows a marketing team to isolate the effect of a marketing action and measure the effectiveness or otherwise of that individual marketing action. Honest marketing teams will only take credit for the incremental effect of their campaign.

Why train on one metric, and measure on another? There are several reasons:

Reason 1: Propensity modelling is relatively easy, uplift modelling is hard

Building a propensity model is a classic supervised machine learning problem. You start with a set of labels that indicate 1 for a customer buying the product, and 0 for a customer not buying the product. You then engineer a set of data features about each customer that have some correlation with purchasing behaviour. With the propensity model trained on the features, you can then feed in a new list of customers to get a set of predictions on who is most likely to buy the product.

Gutierrez (2016) states that uplift modelling is both a machine learning problem and a causal inference problem. Let's break that statement down:

Causal inference: We want to estimate the difference between two outcomes that are mutually exclusive for an individual. In theory, we want to know what would happen if you didn't receive the phone call, and if you did receive it. This is clearly impossible to know, so we need to rely on randomised experiments. Customers must be randomly assigned to either receive a phone call (treatment group) or not receive a phone call (control group).

Machine learning: The two-model approach to uplift modelling will take results from the treatment group and control group, and use them to build two separate models. One model predicts purchasing behaviour with no marketing, and the other with marketing. The two models are subtracted from each other to predict the uplift for any given customer.

The Python package pylift provides fast uplift modelling implementations and evaluation tools. It also heavily leverages the optimisations of other packages, including xgboost, sklearn, pandas, matplotlib, numpy, and scipy. One way pylift achieves this by transforming the data using the Transformed Outcome method, detailed by Athey (2016). The details of the method are outside the scope of this post, but the pylift documentation gives a good introduction.

Reason 2: You will not have the right data at the beginning

The need to rely on a set of data from a randomised experiment makes things difficult in large organisations. Essentially, you need to tell the marketing team that they are not allowed to market to a large set of customers for your experiment, and that the ones they do market to must be randomly selected. Justifying such an exercise can be tricky, as it is possible the company will experience a reduction in sales in that quarter due to the data gathering activity.

Is uplift modelling worth it?

With challenges around data gathering, modelling and implementing a new approach, should you pursue uplift modelling? When successful, an uplift model will:

• Increase profits by identifying customers who are most likely to buy the product after receiving marketing;

• Reduce costs by eliminating prospects who are 'lost causes' (will never buy) or 'sure things' (will buy whether you market to them or not); and

• Avoid waking the 'sleeping dogs' who may churn or be put off making a purchase because of the marketing.

A more subtle and less measurable impact is that uplift modelling should improve customer satisfaction. Advertising platforms like Facebook and Instagram exploit vast volumes of data to deliver precisely targeted marketing. Customers will begin to expect the same level of targeted marketing from all of the businesses they interact with. Uplift modelling can deliver.