← Series by topic

Generalized-Linear-Models

  1. [Generalized linear models] Deriving GLMs from the exponential family (Rewritten from an earlier blog post)

    Derive generalized linear models (GLMs) from the exponential family. Differentiate the normalization condition to obtain means and variances, and compare normal, Bernoulli, and Poisson distributions. Derive iteratively reweighted least squares (IRLS) from maximum likelihood with a canonical link, implement Poisson regression on artificial data in Python, and check the result against SciPy optimization.

  2. [Generalized linear models] What is a canonical link? (Rewritten from an earlier blog post)

    A canonical link makes the linear predictor of a generalized linear model equal to its natural parameter. Derive the identity, logit, and log links from normal, Bernoulli, and Poisson distributions, and plot their relations to the mean. Estimate regression coefficients by maximum likelihood in Python, explain why the gradient and Hessian simplify, and compare with a probit link.