Logistic Regression
Logistic Regression: Modelling Binary Outcomes
Logistic regression is used when the response variable Y is binary (e.g., success/failure, pass/fail) rather than continuous, so an ordinary straight-line model can predict impossible values below 0 or above 1, especially when extrapolating, and does not enforce the probability bounds.
Instead of modelling the probability directly as a linear function of the predictors, logistic regression models the log-odds (the logit) as linear.
The Model
For a single predictor x:
Solving for p gives the S-shaped logistic (sigmoid) curve:
The coefficient β1 is the change in the log-odds for a one-unit increase in x. Exponentiating it, eβ1, gives the odds ratio, the multiplicative change in the odds of success per unit increase in x.
Worked Example
Suppose a model for passing a certification exam based on hours studied x is ln(p/(1−p))=−4+0.8x. At x=5: the log-odds equal 0, so p=0.5, the point where a predicted pass and fail are equally likely. At x=8: the log-odds equal 2.4, so p=1/(1+e−2.4)≈0.92, a much higher predicted probability.
The fitted logistic model ln(p/(1−p))=−4+0.8x produces an S-shaped probability curve. The dashed lines mark the point where the predicted probability equals 0.50, which occurs at x=5 hours studied.
Predicted Probability of Passing vs Hours Studied
The figure shows a fitted logistic regression model ln(1−pp)=−4+0.8x, where x is hours studied and p is the probability of passing. Which statement correctly interprets the coefficient 0.8?
The fitted logistic model ln(p/(1−p))=−4+0.8x produces an S-shaped probability curve. The dashed lines mark the point where the predicted probability equals 0.50, which occurs at x=5 hours studied.
Predicted Probability of Passing vs Hours Studied
Using the same fitted logistic regression model illustrated in the graph, ln(1−pp)=−4+0.8x, what is the predicted probability that a student who studies x=7.5 hours passes the exam? Give your answer to two decimal places.
Give your answer in probability.
The fitted logistic model ln(p/(1−p))=−4+0.8x produces an S-shaped probability curve. The dashed lines mark the point where the predicted probability equals 0.50, which occurs at x=5 hours studied.
Predicted Probability of Passing vs Hours Studied