Fall 2025: Random Matrix Theory in Data Science and Statistics (EN.553.796)

Course Description

This is a first course in random matrix theory, the study of the eigenvalues and eigenvectors of matrices with random entries that is foundational to high-dimensional statistics and data science. Aside from the main ideas and modern applications of random matrices, a key goal will be to introduce you to the main concepts of probability in high dimensions: concentration of measure, the geometry of high-dimensional spaces and convex sets, Gaussian measure, and sharp transitions and threshold phenomena. The following is a (very) tentative ordered list of specific topics to be covered:

1. Gaussian matrices and dimensionality reduction

  • Geometric method of analysis with concentration inequalities
  • Invariance and relation to random projection
  • Application: Johnson-Lindenstrauss transform and dimensionality reduction
  • Application: Compressed sensing

2. Classical theory of i.i.d. random matrices

  • Moment method for eigenvalue limit theorems
  • Semicircle law for Wigner matrices
  • Marchenko-Pastur law for Wishart matrices
  • Elements of universality
  • Elements of free probability theory
  • Application: Neural networks and random optimization landscapes
  • Application: Covariance estimation

3. Spiked matrix models and principal component analysis (PCA)

  • Resolvent method for eigenvalue limit theorems and further applications to eigenvectors
  • Baik—Ben Arous—Péché (BBP) phase transition in the performance of PCA
  • Computational challenges and statistical-computational gaps in PCA
  • Application: Non-linear improvements to PCA
  • Application: Community detection

4. Matrix concentration inequalities

  • General concentration inequalities and applications to random matrix norms (bounded differences, Lipschitz concentration, etc.)
  • General-purpose bounds on expected random matrix norms (matrix Chernoff, Bernstein, etc.)
  • Non-commutative Khintchine inequality and its exciting recent extensions
  • Application: Randomized numerical linear algebra
Contact & Office Hours

I am the instructor of this course, Tim Kunisky, and the teaching assistant is AMS PhD student Yue Wu.

The best way to contact us is by email, at kunisky [at] jhu.edu and ywu166 [at] jhu.edu, respectively. Our office hours are as follows:

  • TA: Mondays 3:30-4:30pm, Wyman S425 (tutorial room)
  • Me: Fridays 12:00-1:00pm, Wyman N438 (my office)
Schedule

Class will meet Tuesdays and Thursdays, 12:00pm to 1:15pm in Bloomberg 176.

Below is a tentative schedule, to be updated as the semester progresses.

Date Details
Week 1
Aug 26 1. Course logistics. Random vector theory. Properties of Gaussian random vectors. Concentration inequalities.
Aug 28 Proof of concentration of Gaussian random vector norms. Consequences for how to think about high-dimensional geometry. Multiplying by a Gaussian matrix: what does it do? The Gaussian process viewpoint.
Week 2
Sep 2 3. Random matrices for dimensionality reduction: the Johnson-Lindenstrauss transform and lemma. "First moment method" proof technique using union bounds. Sketch of applications and related topics: faster variants, lower bounds. Extending the first moment method to uncountable problems.
Sep 4 4. Concentration of singular values of short fat matrices. Interpretation as a "matrix concentration" inequality. "Geometric method" for random matrix analysis. Discretizing matrix norms with epsilon nets. Non-constructive proof of existence of good epsilon nets.
Week 3
Sep 9 5. Application: compressed sensing with random sensing matrices and the null space and restricted isometry properties.
Sep 11 6. Singular vectors of Gaussian matrices. Algebraic method for almost sure distinctness of singular values and eigenvalues. Invariance of matrix distributions.
Week 4
Sep 16 7. Finish up singular vectors of Gaussian matrices. Haar measure on orthogonal groups and Stiefel manifolds. First steps towards eigenvalue and singular value limit theorems. Empirical description of limiting singular value distributions of rectangular matrices. Statistical implications for covariance estimation. Definition of convergence of empirical spectral distribution.
Sep 18 8. Weak convergence of random measures. The moment method for proving eigenvalue limit theorems. Carleman's criterion for distributions determined by moments. Review of moment method proof of central limit theorem. Time permitting, first calculations towards Wigner's semicircle limit theorem.
Lecture Notes and Materials

You do not need to buy any books for this course. You can find my lecture notes here. If you want to look ahead to future topics, you can look at last year's notes here, but be advised that the topics covered this year might differ slightly. If you notice typos in either set of notes, please let me know.

The following are books or lecture notes that cover some similar material and might be useful to you in addition to my notes:

Grading

Grades will be based on a small number of written homework assignments, class participation, and a final project concerning a recent research paper, open problem, or topic of interest related to the material we cover.

Assignments

Homework will be posted here, and is to be submitted through Gradescope (see Canvas announcements for details). Please try to talk to me in advance if you need more time for an assignment.

Assigned Due Link
Sep 8 Sep 22 Assignment 1
Final Project

Your final project is to do one of the following on a topic related to the content of this course: (1) read and digest a paper and present its content in your own words and style, with some elaboration that is not present in the original paper; (2) perform an interesting computational experiment motivated by something we have seen in class or something you read in a paper and report in detail on the results and their interpretation; or (3) for the intrepid, find an open problem related to something we have seen in class, try to work on it, and report your findings.

The project submission will have two parts:

  • A short in-class presentation during the final exam period.
  • A short written report, also due during the final exam period.

Further details on both components will be posted later in the semester.

The following are reasonable categories from which to choose a topic, along with a few references you might look into. You are also welcome to choose a topic of your own, provided that you describe it and its relevance to the course convincingly on Assignment 2.