Derive the Rayleigh quotient as a weighted average of eigenvalues and find its maximum and minimum. Explain why maximizing projected variance in PCA and minimizing ground-state energy in quantum mechanics share the same form. Use Python with two-dimensional data and a two-level system to plot how direction and relative phase change the quotient.
Derive statistical and machine-learning calculations from the common exponential-family form. Use Bernoulli, Poisson, and normal distributions to connect log-partition derivatives, sufficient statistics, and maximum likelihood. Compute beta-prior updates and logistic regression in Python, and distinguish convexity of the loss from the existence of a finite estimate.
Derive the Poisson equation in semiconductor electrostatics, heat transfer, fluids, and gravity using gradients and divergence. Compare source signs and boundary conditions for electric potential, steady temperature, pressure, and gravitational potential. Use Python to compare one-dimensional finite-difference solutions and errors, then check that a two-dimensional pressure correction removes velocity divergence on a periodic domain.
Derive why eigenvalue problems appear in PCA, vibration analysis, quantum mechanics, and graphs from optimization and time-evolution equations. Use small examples to show eigenvalues as variances, squared angular frequencies, energies, and variation across edges. Compute principal components, vibration modes, a two-state system, and graph partitions in Python, checking residuals and conservation laws.
Derive the normal equations shared by least-squares methods and compare their roles in physics, statistics, and machine learning. Estimate a spring constant from artificial measurements, examine precision-weighted estimation and maximum likelihood with normal errors, then use Python polynomial regression to choose Ridge regularization on validation data and evaluate predictions on unseen data.