Introduction
Diagonal matrix is one of the most simplest matrices. It’s eigenvalues are just the diagonal values and the eigenvectors are just the natural bases vectors .
Diagonalizable matrix, a matrix similar to a diagonal matrix, is also benefited from the simplicity of a diagonal matrix.
In this blog post, I would like to quickly discuss matrix diagonalization.
Definition of Diagonalizable Matrix
An matrix is diagonalizable if it is similar to a diagonal matrix: that is, if there exits an invertible matrix and a diagonal matrix such that .
Diagonalization Theorem
Naturally derived from the definition of diagonalizable matrix, we have the diagonalizable theorem.
An matrix is diagonalizable if and only if has linearly independent eigenvectors.
Proof
Suppose is diagonalizable, , we then have . Because is a diagonal matrix, each column vector of , , must be an eigenvector of and the diagonal value of , , is the corresponding eigenvalue.
Because the matrix is invertible, the column vectors of must be linearly independent.
Suppose has linearly independent eigenvectors which can form an invertible matrix , their corresponding eigenvalues can form a diagonal matrix . By the definition of eigenvalue and eigenvector, .
Because the matrix is invertible, and is diagonalizable.
This concludes the proof.
Algebraic and Geometric Multiplicity
Let be an be matrix, and let be an eigenvalue of .
The algebraic multiplicity of is its multiplicity as a root of the characteristic polynomial of .
The geometric multiplicity of is the dimension of the -eigenspace.
Let be a square matrix and let be an eigenvalue of . Then
The complete proof to this is skipped in this article.
Diagonalization Theorem Variant
There is a variant of diagonalization theorem stated in the language of multiplicities.
Let be an matrix. The following are equivalent:
- is diagonalizable.
- The sum of the geometric multiplicities of the eigenvalues of is equal to .
- The sum of the algebraic multiplicities of the eigenvalues of is equal to , and for each eigenvalue, the geometric multiplicity equals the algebraic multiplicity.
Proof
The proof shows .
If is diagonalizable, has linearly independent eigenvectors. This suggests that the sum of the geometric multiplicities of the eigenvalues of is at least .
Because the geometric multiplicity of a eigenvalue is not larger than the algebraic multiplicity of the eigenvalue, the sum of the geometric multiplicities of the eigenvalues of is not larger than the sum of the algebraic multiplicities of the eigenvalues of .
Because the sum of the algebraic multiplicities of an matrix is at most , the sum of the geometric multiplicities of the eigenvalues of is not larger than .
Therefore, if is diagonalizable, the sum of the geometric multiplicities of the eigenvalues of must be equal to .
Because the sum of the algebraic multiplicities of the eigenvalues of is at least the sum of the geometric multiplicities of the eigenvalues of and at most , if the sum of the geometric multiplicities of the eigenvalues of is equal to , this suggests that the sum of the algebraic multiplicities of the eigenvalues of must be .
Suppose there is at least one eigenvalue whose geometric multiplicity does not equal to the algebraic multiplicity. Because the geometric multiplicity of is less than but not equal to the algebraic multiplicity of now, and the sum of the algebraic multiplicities of the eigenvalues of is at least the sum of the geometric multiplicities of the eigenvalues of , which is , the algebraic multiplicities must be at least . However, the the sum of the algebraic multiplicities of the eigenvalues of cannot be larger than . This raises a contradiction. Thus, for each eigenvalue, the geometric multiplicity equals the algebraic multiplicity.
Therefore, if the sum of the geometric multiplicities of the eigenvalues of is equal to , then the sum of the algebraic multiplicities of the eigenvalues of is equal to , and for each eigenvalue, the geometric multiplicity equals the algebraic multiplicity.
Because the sum of the algebraic multiplicities of the eigenvalues of is equal to , and for each eigenvalue, the geometric multiplicity equals the algebraic multiplicity, then the sum of geometric multiplicity is .
Suppose the distinct eigenvalues are , the geometric multiplicities corresponding to the distinct eigenvalues are and , the linearly independent eigenvectors corresponding to the -eigenspace are . is a basis for the -eigenspace, which can just consist of . The basis of forms a matrix . We will show that all the column vectors from the matrices are linearly independent, i.e., the equation
has no non-zero solutions.
Suppose all the column vectors from the matrices are linearly dependent. There exists at least one such that . Because are linearly independent, . Thus,
Multiplying the above equation by on both ends, we have
Using the property of eigenvector and eigenvalues, we have
Thus, we have
Because is not zero for and thus the following linear equation has non-zero solutions.
Because , cannot be all zeros.
For a subset whose which are not zero, suppose , the following linear equation has non-zero solutions.
Notice that , if not zero, is an eigenvector of whose corresponding distinct eigenvalue is . Because eigenvectors with distinct eigenvalues are linearly independent, the zero solution is the only solution to the above linear equation.
This contradicts our previous finding that the above linear equation has non-zero solutions.
Therefore, all the column vectors from the matrices are linearly independent, has linearly independent eigenvectors, and is diagonalizable.
This concludes the proof.
As we have discussed in the previous article “Matrix Similarity”, the vector coordinate transformation using a diagonalizable matrix , where , is just same as doing coordinate scaling in the new bases coordinate system where which are the columns of the matrix , i.e., the eigenvectors of the matrix , and the scaling factors are the corresponding eigenvalues for each eigenvector .
Matrix with Distinct Complex Eigenvalue
If the eigenvalues of an matrix are distinct complex numbers, the eigenvectors of are also linearly independent. Thus, is diagonalizable. This time, the diagonal matrix that the matrix is similar to is a diagonal matrix with the distinct complex eigenvalues of on the diagonal. The coordinate scaling factor, however, is not real but complex. This complex scaling sometimes does not have a good geometric interpretation. Therefore, it’s not quite common to diagonalize a matrix with complex eigenvalues.
As we will see in my later articles, the matrices with complex eigenvalues are similar to real-valued rotation-scaling matrices or real-valued block-diagonal matrices. These matrices are more commonly used in practice and have much better geometric interpretations.
Matrix Diagonalization
To diagonalize a diagonalizable matrix , where , finding the invertible matrix and the diagonal matrix is simply finding the eigenvectors and the eigenvalues of the matrix .
Matrix Eigendecomposition
Matrix eigendecomposition and diagonalization are the same.
References