Data science simplified: Interactions and limitations of regression models

In the last few articles of this series, we discussed regression models at length. Fernando, our data scientist who wants to buy a car, has built a multivariate regression model which takes the following shape:

price = -55089.98 + 87.34 engineSize + 60.93 horsepower + 770.42width

Recall that a multivariate regression model assumes independence between the independent predictors. It treats horsepower, engine size, and width as if they are not related.

However, in practice, variables are rarely independent. What if there are relations between horsepower, engine size, and width? Can these relationships be measured?

This article will address this and explain the concept of interactions.

The concept

The independence between predictors means that if one predictor changes, it has an impact on the target. But this impact has no relation to the existence or changes in other predictors. The relationship between the target and the predictors is additive and linear.

Let us take an example to illustrate this. Again, Fernando’s equation is:

price = -55089.98 + 87.34 x engine size + 60.93 x horsepower + 770.42 width

Here, we can interpret that a change in engine size of one unit changes the price by US$87.34.

However, this interpretation never takes into consideration that engine size may be related to the width of the car. Could it be that the wider the car, the bigger the engine?

A third predictor captures the interaction between engine and width. This third predictor is called the interaction term.

With the interaction term between engine size and width, the regression model takes the following shape:

price = β0 + β1. engine size + β2. horsepower + β3. width + β4. (engine size. width)

The part of the equation β1. engine size + β3. width is called the main effect.

The term engine size x width is the interaction term.

Model building

Limitations of regression models

Conclusion