Is your competition tough? Are your competitors disrupting the industry? Are you struggling to keep profits growing? In this new world survival means looking for better ways to do things. How do we lower costs? How do we get more customers? How do we increase profits? Everyone is looking for those insights that will help them succeed and even exceed in their business. Many companies are succeeding and exceeding by leveraging their data. Why are they doing that and what does that actually mean?
To understand, let’s look at one example. Suppose you have built a rocket ship to take people to the moon. Now you want to let active, interested buyers know about a deal you are offering so that they will buy a trip. It costs you time, money, and reputation to send that offer to people who don’t want it, and you lose revenue if you don’t send it to the people who do want it. To narrow your audience, your goal is to find the characteristics of interested buyers and send it only to them.
But data is messy, and data about people is even messier. Think of all the forms you have filled out. Have you filled them out in the same way every time? And your buying habits are not the same from time to time because what motivates you for one purchase may not be what motivates you for the next. To find the insight you have to sift through the noise in the data and find the signal that best describes what you’re looking for. That’s what data science – data modeling, analytics, and statistics – does.
Data from purchasable demographics and results from past mailings are combined for the raw material on which to build statistical models. These models (such as logistic regression, decision trees and neural nets) find the best relationship between the inputs and the outputs that we are interested in. Data science gives us an “equation” that models the relationship along with a measure of the uncertainty in that equation. And there is always uncertainty. Remember how messy data and people are!
When you have that equation and measure of uncertainty, you can put new data through the equation and get your predicted output. Voila! The equations and predictions deliver the insights that everyone is looking for, and you can use the predicted outputs to make our decisions. In our example, the equation predicts the probability of the recipient responding favorably to your 20 percent discount on flights to the moon. (Remember that you’re not trying to manipulate people into responding favorably; you’re looking to give them what they already want.)
The model will only give you an estimate of the probability that the person will respond favorably. For example, you might find that a person with certain demographic, socioeconomic, and behavioral characteristics has a 90 percent chance of responding favorably to the 20 percent. Even if you have no uncertainty, then there is still a 10 percent chance of the offer ending up in the physical or digital trash.
Each time you make a business decision and act upon it, you can watch the results. If the results come out the way you predicted, then the model worked well. If the results are contrary to what the model predicted, then something in the model was wrong. Either way, you can use that information to improve the model next time.
For example, if your model says that with certain characteristics your target buyer has a 90 percent chance of responding favorably, but they don’t, then when you rerun the analysis, that new result will be used to improve the results. These analytical systems can be set up to provide feedback and automatically rerun models at regular time intervals. You’ll find that the most successful companies use analytical feedback loops to improve their performance.
The Realities of Data Models
Remember, of course, that we live in a messy human world. Often, analytical results are one piece of a larger decision. Budget constraints, legal concerns, human abilities, and many other factors may impact how a decision is made. There’s nothing wrong with that.
And it may make sense to include the human factor as input to the model. For example, suppose we are predicting sales in a market, you might use past behavior, prices of competing products, and correlated variables – like the weather’s effect on sales of icy cold lemonade or steaming hot cappuccinos – to make the prediction. However, a high-performing salesperson could also have their own impression about what the future sales will be. Their subjective prediction could be used as another input for the model. If they make good predictions, then it will help the prediction. If they don’t, then the model will show their poor predictive capability.
For pure prediction you are not very concerned about the relationship between the inputs and the outputs. You probably won’t care about the strength or the direction (positive or negative) – only whether an input helps you predict or not. Other times the relationships are very important and they give you a prescriptive model. You want to know what inputs are most important and whether they positively or negatively impact the output.
Let’s look at another example. Suppose you’re using data about the raw materials and processing characteristics to predict the quality of a batch of widgets. It’s not enough to know what the quality currently is. You want to know how to increase the quality. You want to prescribe the characteristics needed to provide the best quality you can at the right price. A data scientist might find that a certain raw material doesn’t impact product quality as much as the company thought. More tests might show that show a lower-cost supplier gives the same quality result – and you’ve uncovered a way to reduce production costs. Data scientists exploring the data can find key insights that can make a company a lot of money here.
An important caveat: the more experienced the person building the model is, the more likely the model will be a good representation of reality. However, with analytical software so easy to use now, it can be tempting to build advanced models without being well trained. But a bad model can waste money, produce bad products, or even be dangerous, depending on what you’re modeling (and deciding to do with the information those models provide).
With accurate, robust models, companies can turn data into insights that improve just about every decision they make, including how to increase profits. The right data scientist can be worth his or her weight in gold.