Data science simplified: Understanding logistic regression

Base photo credit: Stanley Dai.

Pradeep is a star contributor for Tech in Asia and publishes exclusive, high-value content that serves the Asian tech community. Read more from star contributors here.

In the previous article of this series, we discussed classifiers, its categories, and how they are evaluated. We’ve also previously discussed regression models in depth. In this post, we’ll dwell a little deeper in how regression models can be used for classification tasks.

Logistic regression is a widely used regression model for classification tasks. As usual, we’ll use an example to discuss this concept.

Let’s say a bank wants to build a model that predicts which of their customers will default on their loans. Their dataset is as follows:

The features in this dataset that will help us build the model are:

• Customer ID: A unique string to identify each customer
• Credit score: A value between 0 and 8,000 that indicates the riskiness of the borrower’s credit history
• Loan amount: The amount that the customer either completely paid off or defaulted on
• Years in current job: The number of years the customer has been in their current job
• Years of credit history: The years since the first entry in the customer’s credit history
• Monthly debt: The customer’s monthly payment for their existing loans
• Number of credit problems: The number of credit problems that are present in the customer’s record
• Is default: If the customer has defaulted, then this number will be 1. If not, it will be 0.

This is a classification problem.

Why not use a plain old regression model?

Logistic regression is an avatar of the regression model and transforms the regression model into a classifier. But first, let’s understand why a plain regression model won’t work as a classifier.

The target “is default” has a value of 0 or 1. So, we can reframe this as a probability, as follows:

• If the probability of default ≥ 0.5, then the customer will default (i.e. is default = 1).
• If the probability of default

Recall our discussion on linear regression models where we defined a dependent variable (y) which was a function of independent variables. For the sake of simplicity, let’s assume that we have only one independent variable (x). The equation becomes:

Sigmoid

Logistic regression

Conclusion