366 Logistic regression and hyperplanes

366 Logistic regression and hyperplanes

Logistic regression is a statistical method for predicting the probability of an event occurring and is actually a model used for classification problems. A typical application is to predict whether an email belongs to one of two alternatives, for example, whether an email is spam or not, or whether a patient is sick or not. Despite the name "regression," this is not a regression that directly predicts a numerical value, but rather a model used to classify the final class (0 or 1) through probability.

In this logistic regression, the input features are summed with weights, or so-called linear combination, and the result is passed through an S-shaped function called a sigmoid function to convert it into a probability between 0 and 1. The classification is then determined by whether this probability exceeds a certain threshold (often 0.5). In this way, logistic regression divides the data into two groups by drawing certain boundaries in the feature space.

The concept of a "hyperplane" refers to a boundary plane for dividing the feature space, which is a straight line in two-dimensional space and a plane in three-dimensional space; similar boundaries exist in four or more dimensions, all of which are collectively called "hyperplanes. In logistic regression, the hyperplane is the set of points where the weighted linear function is zero, and is the boundary of classification. For example, "1 above this hyperplane, 0 below it," serves to dichotomize the data.

As a concrete example, let us assume that we want to predict whether a person will be approved for a credit card using two characteristics, age and income. Logistic regression maps the data into this two-dimensional space and finds the single straight line (i.e., hyperplane) that best divides them. People on one side of this line are judged to pass, while those on the other side are judged to fail.

However, because logistic regression is a linear model, only simple hyperplanes such as straight lines or planes can be drawn. Therefore, in cases where the data are complex and intertwined, other models that can learn more flexible boundaries, such as support vector machines (SVMs), neural networks, or other nonlinear models, are needed. The strength of logistic regression lies in its simplicity and ease of interpretation, making it a particularly effective method in situations where one wants to explicitly understand the relationship between features and results.