"Theory of Matrices Class Notes" Webpage

Theory of Matrices Class Notes
Matrix Algebra: Theory, Computations, and Applications in Statistics, by James E. Gentle, Springer (2007, 2017). Gentle's Matrix Algebra book

Gentle's Matrix Algebra book, 2nd edition

The catalog description of Theory of Matrices (MATH 5090) is very brief: "Vector spaces, linear transformations, matrices, and inner product spaces." In light of our department's name, "The Department of Mathematics and Statistics," this class will try to branch these two areas. This is why we use a textbook in the "Springer Texts in Statistics" series. The class covers the first 5 chapters of the textbook, Matrix Algebra: Theory, Computations, and Applications in Statistics, which is the "Theory" part of the book (Chapters 6 and 7 are in the "Linear Algebra" part of the book, but actually address more computational material). The author of the text states (see page viii) "...the presentation is informal; neither definitions nor facts are highlighted by words as 'Definition', 'Theorem', 'Lemma', and so forth. ... Most of the facts have simple proofs, and most proofs are given naturally in the text. No 'Proof' and 'Q.E.D.' or '■' appear to indicate beginning and end." We will follow a different approach and will be introducing exactly the definition/theorem/proof approach in these class notes. Results contained in Gentle's book are numbered as chapter.section.number (such as "Theorem 3.2.1") and labeled results from other sources are numbered by the chapter, section, and a letter (such as "Theorem 3.2.A"). Most of the notes are based on the first (2007) edition of the text book. The notes based on the second (2017) edition of the book are marked below with an asterisk. We will not take as much of a theoretical approach as I do in my other graduate-level classes. For example, we will deal with matrices with real entries and not deal with the entries as elements of a field nor with the more abstract algebra properties of matrices. A more abstract approach to linear algebra can be found in the online notes for Linear Algebra (Graduate Level), based on Thomas Hungerford's Algebra (Springer-Verlag, 1974).

Copies of the classnotes are on the internet in PDF format as given below. The "Proofs of Theorems" files were prepared in Beamer and they contain proofs of results which are particularly lengthy (shorter proofs are contained in the notes themselves). The "Printout of Proofs" are printable PDF files of the Beamer slides without the pauses.

Chapter 1. Basic Vector/Matrix Structure and Notation.
Chapter 2. Vectors and Vector Spaces.
Chapter 3. Basic Properties of Matrices.
Chapter 4. Vector/Matrix Derivatives and Integrals.
Chapter 5. Matrix Transformations and Factorizations.
Chapter 6. Solution of Linear Systems.
Chapter 7. Evaluation of Eigenvalues and Eigenvectors.
Chapter 8. Special Matrices and Operations Useful in Modeling.
Chapter 9. Selected Applications in Statistics.
Chapter 10. Numerical Methods.
Chapter 11. Numerical Linear Algebra.
Chapter 12. Software for Numerical Linear Algebra.
Appendix A. Notation and Definitions.
Appendix B. Solutions and Hints for Selcted Exercises.

Part I. Linear Algebra

Chapter 1. Basic Vector/Matrix Structure and Notation. Chapter 1 notes

Study Guide 1.

Chapter 2. Vectors and Vector Spaces.

Section 2.1. Operations on Vectors. Section 2.1 notes

Supplement. Proofs of Theorems in Section 2.1. Beamer file of Section 2.1 proofs
Supplement. Printout of the Proofs of Theorems in Section 2.1. Print file of Section 2.1 proofs

Section 2.2. Cartesian Coordinates and Geometrical Properties of Vectors. Section 2.2 notes

Supplement. Proofs of Theorems in Section 2.2. Beamer file of Section 2.2 proofs
Supplement. Printout of the Proofs of Theorems in Section 2.2. Print file of Section 2.2 proofs

Section 2.3. Centered Vectors and Variances and Covariances of Vectors. Section 2.3 notes

Supplement. Proofs of Theorems in Section 2.3. Beamer file of Section 2.3 proofs
Supplement. Printout of the Proofs of Theorems in Section 2.3. Print file of Section 2.3 proofs

Section 2.3 Supplement. Centered Vectors and Variances and Covariances of Vectors: Formulas. Formulas for Centered Vectors, Variances, and Covariances of Vector
Study Guide 2.

Chapter 3. Basic Properties of Matrices.

Section 3.1. Basic Definitions and Notation. Section 3.1 notes

Supplement. Proofs of Theorems in Section 3.1. Beamer file of Section 3.1 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.1. Print file of Section 3.1 proofs

Section 3.2. Multiplication of Matrices and Multiplication of Vectors and Matrices. Section 3.2 notes

Supplement. Proofs of Theorems in Section 3.2. Beamer file of Section 3.2 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.2. Print file of Section 3.2 proofs

Section 3.3. Matrix Rank and the Inverse of a Full Rank Matrix. Section 3.3 notes

Supplement. Proofs of Theorems in Section 3.3. Beamer file of Section 3.3 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.3. Print file of Section 3.3 proofs

Section 3.4. More on Partitioned Square Matrices: The Schur Complement. Section 3.4 notes

Supplement. Proofs of Theorems in Section 3.4. Beamer file of Section 3.4 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.4. Print file of Section 3.4 proofs

Section 3.5. Linear Systems of Equations. Section 3.5 notes

Supplement. Proofs of Theorems in Section 3.5. Beamer file of Section 3.5 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.5. Print file of Section 3.5 proofs

Section 3.6. Generalized Inverses. Section 3.6 notes

Supplement. Proofs of Theorems in Section 3.6. Beamer file of Section 3.6 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.6. Print file of Section 3.6 proofs

Section 3.7. Orthogonality. Section 3.7 notes

Supplement. Proofs of Theorems in Section 3.7. Beamer file of Section 3.7 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.7. Print file of Section 3.7 proofs

Section 3.8. Eigenanalysis; Canonical Factorizations. Section 3.8 notes

Supplement. Proofs of Theorems in Section 3.8. Beamer file of Section 3.8 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.8. Print file of Section 3.8 proofs

Section 3.9. Matrix Norms. Section 3.9 notes

Supplement. Proofs of Theorems in Section 3.9. Beamer file of Section 3.9 proofs
Supplement. Printout of the Proofs of Theorems in Section 3.9. Print file of Section 3.9 proofs

Section 3.10. Approximation of Matrices. Section 3.10 notes
Study Guide 3.

Chapter 4. Vector/Matrix Derivatives and Integrals.

Section 4.1. Basics of Differentiation. Section 4.1 notes
Section 4.2. Types of Differentiation. Section 4.2 notes

Supplement. Proofs of Theorems in Section 4.2. Beamer file of Section 4.2 proofs
Supplement. Printout of the Proofs of Theorems in Section 4.2. Print file of Section 4.2 proofs

Section 4.3. Optimization of Functions (Partial). Section 4.3 notes
Section 4.4. Multiparameter Likelihood Functions. Section 4.4 notes
Section 4.5. Integration and Expectation. Section 4.5 notes

Supplement. Proofs of Theorems in Section 4.5. Beamer file of Section 4.5 proofs
Supplement. Printout of the Proofs of Theorems in Section 4.5. Print file of Section 4.5 proofs

Study Guide 4.

Chapter 5. Matrix Transformations and Factorizations.

Section 5.1. Transformations by Orthogonal Matrices. Section 5.1 notes
Section 5.2. Geometric Transformations. Section 5.2 notes
Section 5.3. Householder Transformations (Reflections). Section 5.3 notes

Supplement. Proofs of Theorems in Section 5.3. Beamer file of Section 5.3 proofs
Supplement. Printout of the Proofs of Theorems in Section 5.3. Print file of Section 5.3 proofs

Section 5.4. Givens Transformations (Rotations). Section 5.4 notes
Section 5.5. Factorizations of Matrices. Section 5.5 notes
Section 5.6. LU and LDU Factorizations. Section 5.6 notes

Supplement. Proofs of Theorems in Section 5.6. Beamer file of Section 5.6 proofs
Supplement. Printout of the Proofs of Theorems in Section 5.6. Print file of Section 5.6 proofs

Section 5.7. QR Factorization. Section 5.7 notes

Supplement. Proofs of Theorems in Section 5.7. Beamer file of Section 5.7 proofs
Supplement. Printout of the Proofs of Theorems in Section 5.7. Print file of Section 5.7 proofs

Section 5.8. Singular Value Factorization. Section 5.8 notes
Section 5.9. Factorizations of Nonnegative Definite Matrices. Section 5.9 notes

Supplement. Proofs of Theorems in Section 5.9. Beamer file of Section 5.9 proofs
Supplement. Printout of the Proofs of Theorems in Section 5.9. Print file of Section 5.9 proofs