Data science simplified: Selecting the best model

In the last article of this series, we discussed multivariate linear regression models. Fernando created a model that estimated the price of cars based on five input parameters. But, although he did have a better model, he wanted to select the best set of variables for input. This article will elaborate on model selection methods.

Concept

The idea of the model selection method is intuitive. It answers the question “How do you select the right input variables for an optimal model?”

So, how do you define an optimal model? An optimal model is a model that fits the data with the best values for the evaluation metrics.

There can be a lot of evaluation metrics. In the case of multivariate linear regression models (like Fernando’s) the adjusted R-squared is the chosen evaluation metric.

There are three methods for selecting the best set of variables. They are:

• Best subset
• Forward stepwise
• Backward stepwise

Let us dive into the inner workings of these methods.

Best subset

For this method, let us say that we have k variables. The process for the best subset method is as follows:

1. Start with the null model (i.e. the model with no predictors). Let us call this model as M0.
2. Create the model with one variable (M1).
3. Create the model with two variables (M2).
4. Create the model with three variables (M3).
5. And so on. Repeat this process and test all the combinations of predictors for the optimal model.

Forward stepwise

Backward stepwise

Model Evaluation

Conclusion

---