This is why AI models will never survive without humans

Companies often expect that AI models, once implemented, will stay smart and keep working.

Nothing can be further from the truth.

Business leaders often find this disappointing: a machine learning model, which consumed precious time and dollars still needs humans for routine maintenance.

Let’s see why this is the reality today and how companies can plan for it.

How typical ML models work

Let’s say you are a telecom company looking to solve the big problem of customer churn. You want early predictive warnings on the customers who will leave in the coming month.

You duly collect a dozen data feeds about customer demographics, purchases, subscription plans, and service interactions. You then hand over these 100+ attributes from the past few years, which run into millions of data points.

Data scientists try to analyze and make sense of all this data. They then build and engineer models that can predict, say eight of the 10 customers who would eventually leave. When piloted, this works beautifully and lets you focus on the task of retaining these customers. All is well, so far.

Let’s pause now and unwrap this model.

From the 100+ factors of customer attributes that you supplied, you’ll find that the model just uses three to five. After studying the strength of all signals, algorithms usually end up using a tiny set of factors (or maybe just one) that are most related to customer churn.

The entire discipline of machine learning is about identifying those few factors (predictors) and then figuring out their relationship (the formula) to the outcome (target).

Expectation vs reality

Four scenarios that shake the fundamentals of a model

You now get a sense of how fragile the internals of a model can be, in spite of the apparently sophisticated facade. Unfortunately, people often assume that machines have a grounded, comprehensive understanding of the situation.

To be fair, we humans also base complex decisions on a seemingly small set of factors. However, the brain has a superior ability to play with these factors or their relative importance for a decision.

Here are the key scenarios that call for models to get back into the classroom:

What does it take to keep the models smart?

Closing thoughts