Mathematics Numerical computing
[Linear algebra] Understanding least squares as a projection (Rewritten from an earlier blog post)
Explain least squares as the orthogonal projection of an observation vector onto a column space. Derive the normal equations from projection onto a line and check that the projection matrix is symmetric and idempotent. Compare a three-point regression by hand and in Python, plot residual orthogonality, and examine QR factorization and repeated columns with nonunique coefficients.
