Logit / Probit

Motivating Example: Firm Adoption of a New Technology#

Imagine you are studying why some firms adopt a new manufacturing technology and others do not. Your outcome variable is binary: $Y_i = 1$ if firm $i$ adopts, $Y_i = 0$ otherwise. You want to know how firm size, R&D spending, and industry competition affect the probability of adoption.

You could try running OLS — regressing the 0/1 outcome on your covariates. This approach is called the linear probability model (LPM), and it is a reasonable starting point. But it has problems. The predicted probabilities can fall outside $[0, 1]$, the error term is necessarily heteroskedastic, and the marginal effect of a covariate is assumed to be constant regardless of where you are on the probability scale.

Logit and probit models address these problems by modeling the probability through a nonlinear link function that keeps predictions bounded between 0 and 1.

The Problem with OLS on Binary Outcomes#

When you run $Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$ with $Y_i \in \{0, 1\}$, you are modeling:

This equation is the LPM. It works surprisingly well in many cases, especially near the center of the data. But at the extremes, it can predict probabilities below 0 or above 1, which is nonsensical.

Link Functions: The Core Idea#

Both logit and probit model the probability through a nonlinear transformation:

where $G(\cdot)$ is a function that maps any real number to the $(0, 1)$ interval.

• Logit uses the logistic function: $G(z) = \frac{e^z}{1 + e^z} = \Lambda(z)$
• Probit uses the standard normal CDF: $G(z) = \Phi(z)$

Both are S-shaped curves. They are nearly identical in practice — probit is slightly steeper at the center and slightly thinner at the tails. In most applications, they give very similar results.

When Does It Matter Which You Choose?#

In most applications, it does not. The choice between logit and probit rarely changes substantive conclusions. Logit is more common in epidemiology and management because of the convenient odds ratio interpretation. Probit is more common in economics, partly by convention and partly because it connects naturally to latent variable models.

Common Confusions#

The identification strategy for logit/probit is the same as for OLS: you need exogeneity of the regressors. The logit/probit framework does not solve endogeneity problems — it just handles the functional form for binary outcomes.

If your regressors are endogenous, you need an identification strategy (IV, DiD, matching, etc.) combined with the appropriate binary outcome model. For IV with binary outcomes, see the bivariate probit or IV-probit approach. It is also advisable to consider sensitivity analysis to assess how robust your estimates are to potential unobserved confounders.

The Latent Variable Interpretation#

Both models can be motivated by a latent variable $Y_i^*$:

If $\varepsilon_i$ follows a logistic distribution, you get logit. If $\varepsilon_i$ follows a standard normal, you get probit. The firm adopts the technology when the latent net benefit exceeds zero.

Think of the probability curve as a hill. At the bottom (low probability of adoption), even a large change in firm size barely moves the probability — you are pushing against inertia. At the top (high probability), the same is true — most firms have already adopted. The steepest part of the hill is in the middle, around 50% probability. This middle region is where a change in X has the biggest effect on the probability.

This nonlinearity is why marginal effects depend on where you evaluate them. A one-unit increase in firm size might raise adoption probability by 8 percentage points for a mid-sized firm (on the steep part of the curve) but only 2 percentage points for a very large firm (on the flat part).

1# Requires: marginaleffects
2library(marginaleffects)
3
4# --- Step 1: Fit the Logit Model ---
5# glm() with family=binomial(link="logit") estimates via maximum likelihood.
6# Coefficients are in LOG-ODDS units, not probability units.
7logit_fit <- glm(adopt ~ firm_size + rd_spending + competition,
8               family = binomial(link = "logit"), data = df)
9# summary() shows log-odds coefficients, SEs, z-values, and Akaike Information Criterion (AIC)
10summary(logit_fit)
11
12# --- Step 2: Compute Average Marginal Effects (AMEs) ---
13# AMEs translate log-odds coefficients into probability-scale effects.
14# Each AME represents the average change in P(Y=1) for a one-unit change in X,
15# averaged across all observations (accounting for nonlinearity).
16# marginaleffects::avg_slopes() is the modern replacement for the archived margins package.
17ame <- avg_slopes(logit_fit)
18# Output: AME in percentage-point terms — the primary quantity to report
19print(ame)
20
21# --- Step 3: Compute Odds Ratios ---
22# Exponentiate coefficients to get odds ratios: exp(beta).
23# An OR of 1.35 means a one-unit increase in X multiplies the odds by 1.35.
24exp(coef(logit_fit))
25# Confidence intervals for odds ratios (profile likelihood-based)
26exp(confint(logit_fit))
27
28# --- Step 4: Fit Probit for Robustness ---
29# Probit uses the normal CDF as the link function instead of logistic.
30# Results should be substantively similar to logit — showing both
31# demonstrates robustness to the choice of link function.
32probit_fit <- glm(adopt ~ firm_size + rd_spending + competition,
33                family = binomial(link = "probit"), data = df)
34# Compare probit AMEs to logit AMEs — they should nearly agree
35print(avg_slopes(probit_fit))

Pseudo R-Squared#

There is no true $R^2$ for logit/probit. McFadden's pseudo-$R^2$ compares the log-likelihood of your model to a null model (intercept only):

Values above 0.2 are sometimes informally considered indicative of good fit, though interpretation depends on context and there is no universal threshold. Do not compare pseudo-$R^2$ values across different link functions.

Classification Table#

Predict $\hat{Y}_i = 1$ if $\hat{p}_i > c$ (usually $c = 0.5$) and compute the confusion matrix. Report sensitivity (true positive rate), specificity (true negative rate), and overall accuracy. But be cautious: classification accuracy is sensitive to class imbalance.

Hosmer-Lemeshow Test#

Groups observations into deciles of predicted probability and tests whether observed frequencies match predicted frequencies. A significant test suggests poor calibration, but the test has low power and is sensitive to the number of groups.

Three Ways to Report Logit Results#

1. Log-odds coefficients — the raw output. Hard to interpret; mainly useful for checking sign and significance.
2. Odds ratios — $e^{\beta_j}$. "A one-unit increase in X multiplies the odds of Y=1 by $e^{\beta_j}$." Common in epidemiology and management.
3. Marginal effects — the change in probability. Most intuitive. Preferred in economics.

• Linear Probability Model (LPM): If your probabilities are between 0.2 and 0.8 for most observations, the LPM with robust SEs often gives nearly identical average marginal effects. Easier to interpret and to combine with FE or IV.
• Conditional logit (fixed effects logit): For panel data with unit fixed effects. Only uses within-unit variation. See Chamberlain (1980). Unlike logit, probit does not have an analogous conditional MLE that eliminates fixed effects. FE probit suffers from the incidental parameters problem and is inconsistent with fixed T (Wooldridge, 2010).
• Multinomial logit: When the outcome has more than two unordered categories.
• Ordered logit/probit: When the outcome has ordered categories (e.g., strongly disagree to strongly agree).
• Count models: When the outcome is a non-negative integer (number of events), see Poisson / Negative Binomial instead.