Logit Regression and the Geometry of Odds

A linear model for log odds

Logit models the logarithm of the odds as a linear function. The inverse-logit function keeps every fitted probability between zero and one (Train, 2009).

Increasing $x_j$ by one multiplies the odds by $\exp(\beta_j)$. It does not add a constant amount to the probability.

A numerical odds calculation

Let $\log[p/(1-p)]=-1+0.7x$. The coefficient implies an odds ratio of $\exp(0.7)=2.014$.

The marginal effect is largest near $p=0.5$:

Assumptions and implementation

Maximum likelihood requires a specified conditional mean and independent sampling for its usual covariance formula. Robust or clustered covariance estimates can relax variance and dependence assumptions. Complete separation can send one or more estimates toward infinity.

Equal index steps create unequal probability changes because the logistic curve is S-shaped.

From log odds to probabilities

A coefficient $\beta_j$ gives the change in log odds from a one-unit increase in $x_j$, holding other regressors fixed. Exponentiation gives an odds ratio. Probability changes vary with the starting index because the logistic curve is steepest near one-half. In the numerical example, the odds calculation is exact for the stated covariate shift. It should not be described as the same percentage change in probability for every person.

Report predicted probabilities or average marginal effects beside odds ratios. For a binary regressor, use the average discrete change between its two values. State the covariate distribution used for averaging.

Estimation checks and failure modes

Confirm that the outcome uses two documented values and that the modeled event matches the reported event. Inspect cross-tabulations for complete or quasi-complete separation. Under separation, maximum-likelihood coefficients can diverge even when software returns a large finite number. Penalized likelihood can stabilize estimation, but it changes the estimator and requires disclosure.

Check influential observations, overlap across important groups, and calibration of predicted risks. A correct link function still needs a suitable linear index. Splines or interactions may be needed for continuous predictors. Robust or clustered covariance can change uncertainty without repairing misspecified probabilities. When observations come from a complex sample, incorporate the sampling design in both estimation and variance calculations. Avoid interpreting fitted probabilities far beyond the observed predictor support. (Greene, 2018)

Further reading

Train derives logit from extreme-value random utility and explains odds and substitution patterns (Train, 2009). Stock and Watson give an applied comparison with linear probability and probit models (Stock and Watson, 2019).