Linear AlgebraTransposes

The dot product gives us a compact way to express the formula for an entry of a matrix product: to obtain the $(i,j)$th entry of a matrix product , we dot the $i$th row of and the $j$th column of .

However, the matrix product by itself is not quite flexible enough to handle a common use case: suppose we have two matrices and which contain tabular data stored in the same format. For example, suppose that the columns of store the vectorized word counts for a series of emails sent from Alice, while stores vectorized word counts for a series of emails sent from Bob. If we want to calculate the similarity of each of Alice's email to each of Bob's emails, then we want to dot the columns of —not its rows—with the columns of .

Continue

So we define the transpose of a matrix to be the matrix which results from switching the rows and columns of .

Continue

With this definition in hand, we can write the matrix whose entries are the dot products of columns of and as .

Continue

Transpose properties

Let's develop a few properties of the transpose so that we can manipulate matrix expressions involving the transpose. First, we note that the transpose is a linear operator, meaning that whenever is a constant and and are matrices.

Taking the transpose also interacts nicely with matrix multiplication:

import numpy as np

Solution. Since is an matrix, the matrix is .

• : this is an matrix multiplied by a matrix, and if it is not defined. If it is defined, it gives an matrix.
• : this is a matrix multiplied by an matrix and hence is a matrix.
• is an matrix, and is an matrix. If this is not defined. If it is defined, it gives an matrix.

We see that the only matrix product that is always defined, and in fact gives the right dimensions, is the second option. And in fact, we have

in general.

The following block of code checks the equation for a particular random example.

import numpy as np
A = np.random.random_sample((3,7))
B = np.random.random_sample((7,3))
np.allclose((A @ B).T, B.T @ A.T)

A = rand(3, 7)
B = rand(7, 3)
isapprox((A * B)', B' * A')

Continue

Symmetric Matrices

In some applications, a matrix will have the property that its $(i,j)(j,i)$th entry. For example, suppose we have an ordered list of 100 cell phone towers, and we define the matrix whose $(i,j)$th entry is equal to the distance from tower to tower . Such a matrix is said to be symmetric.

Continue

Solution. We have

Here we used that for any matrix , and that for a symmetric matrix . This leaves (3) as the correct answer. (5) is close, but incorrect if .

Continue

In the case where is a matrix and is an for some , then is a matrix, which we may think of as just a number. This means that if and are vectors in then the dot product may be written as . This representation can be useful for manipulating expressions involving dot products:

Continue

Solution. Since we have

Continue

In other words, we may move a matrix which is pre-multiplying one of the vectors in a dot product to the other vector, at the cost of taking its transpose. Let's look at one application of this property whose importance will be explored in subsequent sections.

Solution. We have