Binary response models · Binary regression

Probit Regression and Latent Gaussian Choice

Probit links a binary outcome to a latent index through the standard normal distribution.

From a latent score to an observed choice

Probit starts with an unobserved continuous score. We observe only whether that score crosses zero. This construction gives a smooth probability model for binary outcomes (Stock and Watson, 2019).

$$ y_i^{\mathrm{latent}}=x_i'\beta+\varepsilon_i,\qquad y_i=\mathbf 1\{y_i^{\mathrm{latent}}>0\},\qquad \varepsilon_i\sim N(0,1). $$

Therefore,

$$ \Pr(y_i=1\mid x_i)=\Phi(x_i'\beta). $$

The unit variance fixes the scale of the latent score. Without that normalization, only the ratio of the coefficients to the error scale is identified.

A numerical probability

Suppose the fitted index is $-0.5+0.8x$. The table maps three values of $x$ into probabilities.

$x$	Index	$\Phi(\text{index})$
0	-0.50	0.309
1	0.30	0.618
2	1.10	0.864

The coefficient $0.8$ is an index effect. Its probability effect varies with $x$:

$$ \frac{\partial \Pr(y_i=1\mid x_i)}{\partial x_{ij}}=\phi(x_i'\beta)\beta_j. $$

Assumptions and implementation

Independent observations and a correctly specified conditional index support conventional maximum-likelihood inference. Perfect prediction can make finite estimates unavailable. Software can report coefficients, latent-index predictions, probabilities, or marginal effects. These objects have different scales.

A useful figure would place the normal cumulative distribution beside the three table points. The horizontal axis is the linear index. The vertical axis is the predicted probability.

Interpretation and reporting

A probit coefficient shifts the latent index. It is not a constant probability change. At $x=1$ in the example, the index equals $0.3$, so the local effect is $0.8\phi(0.3)$, about $0.305$.

Report which probability effect answers the research question. A marginal effect at selected covariate values describes those values. An average marginal effect computes each observation's derivative and then averages. A discrete change suits binary regressors.

Predicted probabilities remain between zero and one because $\Phi$ maps every index to that range. Extrapolation can still be weak when new covariate values lie outside the estimation support. Perfect prediction can prevent finite maximum-likelihood estimates.

Reproducible implementation

Verify that the outcome uses exactly two declared codes and document which code denotes one. Add the intended intercept and transformations before fitting maximum likelihood. Check convergence messages and coefficient scale.

Generate linear-index and probability predictions separately. Recalculate several probabilities with the standard normal cumulative distribution. For marginal effects, apply $\phi(x_i'\widehat\beta)\widehat\beta_j$ to each required observation.

Use robust or clustered covariance when the sampling design requires it. Inspect sparse covariate patterns for separation. Save the estimation sample, optimizer status, coefficient table, and effect definition. These records prevent latent-index coefficients from being mislabeled as probability effects (Train, 2009).

Sources and further reading

James H. Stock, Mark W. Watson. 2019. Introduction to Econometrics. Pearson. Source.
Kenneth E. Train. 2009. Discrete Choice Methods with Simulation. Cambridge University Press. doi:10.1017/CBO9780511805271.

About this benchmark task

Status: In the benchmark
Identifier: probit
Family: Binary regression
Software: Stata, R, Python
Source: Benchmark task set

Task statement

Estimate a binary probit of y_bin_probit on x1 and x2 with an intercept. Report coefficient-scale rows const, x1, and x2 with asymptotic standard errors.

Required output

const, x1, x2

Open every recorded run for this task.