Derive statistical and machine-learning calculations from the common exponential-family form. Use Bernoulli, Poisson, and normal distributions to connect log-partition derivatives, sufficient statistics, and maximum likelihood. Compute beta-prior updates and logistic regression in Python, and distinguish convexity of the loss from the existence of a finite estimate.
Derive generalized linear models (GLMs) from the exponential family. Differentiate the normalization condition to obtain means and variances, and compare normal, Bernoulli, and Poisson distributions. Derive iteratively reweighted least squares (IRLS) from maximum likelihood with a canonical link, implement Poisson regression on artificial data in Python, and check the result against SciPy optimization.
A canonical link makes the linear predictor of a generalized linear model equal to its natural parameter. Derive the identity, logit, and log links from normal, Bernoulli, and Poisson distributions, and plot their relations to the mean. Estimate regression coefficients by maximum likelihood in Python, explain why the gradient and Hessian simplify, and compare with a probit link.