## Słownik

Wybierz jedno ze słów kluczowych po lewej stronie…

# Linear AlgebraMatrices

Czas czytania: ~60 min

A matrix is a rectangular array of numbers:

We report the size of a matrix using the convention number of rows by number of columns. In other words, a matrix with rows and columns is said to be an matrix. The matrix above is by .

We refer to the entry in the th row and th column of a matrix as 's (i,j)th entry, and we denote it as . In Python, the (i,j)th entry may be referenced as A[i,j].

We refer to the entry in the th row and th column of a matrix as 's (i,j)th entry, and we denote it as . In Julia, the (i,j)th entry may be referenced as A[i,j].

Matrices are versatile structures with a variety of problem-solving uses. For example,

• A matrix can be thought of as a list of column vectors, so we can use a matrix to package many column vectors into a single mathematical object.

• An matrix can be thought of as a linear transformation from to .

In this section, we will develop both of these perspectives and define some operations which facilitate common manipulations that arise when handling matrices.

Definition (Matrix addition and scalar multiplication)
We define matrix addition for two matrices and entrywise: the sum is , and each entry is defined to be the sum of the corresponding entries in and .

Likewise, the product of a number and an matrix is defined to be the matrix each of whose entries is times the corresponding entry of .

Exercise
Find the value of such that

Note that two matrices are considered equal if each pair of corresponding entries are equal.

The solution is .

Solution. If we look at the middle entry of the bottom row of the two sides of the equation, get

We can see that this equation will hold regardless of the value of . The equation corresponding to the top-right corner is

Solving this equation, we find that . Therefore, if there is a solution to the original matrix equation, it must be . We can then check the remaining four equations to see that is indeed a solution.

## Matrices as linear transformations

One of the most useful ways to think of a matrix is as a concrete representation of a linear transformation. The following definition provides the connection between matrices and maps between vector spaces.

Definition (Matrix-vector multiplication)
If is an matrix and is a column vector in , then is defined to be the linear combination of the columns of with weights given by the entries of .

Example
If and then where , , , and .

As advertised, the transformations described in the definition of matrix-vector multiplication are linear:

Exercise
Suppose that is an matrix. Show that is a linear transformation.

Solution. Suppose has columns and . By definition,

Consider a second vector . We have

Next, let be a constant.

These are the two requirements for a transformation to be considered linear, so is indeed linear.

It turns out that every linear transformation from to can be represented as for some matrix . The entries of the matrix may be obtained from by placing the components of in the first column of , the components of in the second column, and so on.

With this definition of , we have that , and similarly for the other standard basis vectors. Since the equation holds for all in a basis of , we conclude that it holds for all (by the basis equality theorem).

Exercise
Find the matrix corresponding to the linear transformation .

Solution. Based on the first component of the expression for , we find that the first column of the matrix representing is . Similarly, the next two columns are , and . Altogether, we find that the matrix is

Exercise
Suppose that is an matrix and is a vector in with the property that the equation has at least one solution . Show that the solution is unique if and only if the columns of are linearly independent.

The intuition is that provides a recipe for how much of each column of to use to get . If the columns of are linearly dependent, then we can swap out a unit of one of the vectors for some combination of others. This swappability shows that the solution is nonunique.

Solution. If the columns of are not linearly independent, then one of the columns is a linear combination of the columns to its left, say

Therefore, given any solution of , we can obtain another solution by increasing the $k\mathbf{x}$ by and decreasing the first component by , the second by , and so on, up to .

Conversely, if there are distinct solutions and , then . Therefore, the components of provide the weights for a linear combination of the of which is equal to the zero vector.

## Matrix multiplication

With the perspective that matrices should represent linear transformations, it makes sense to define matrix multiplication so that corresponds to composition of the corresponding linear transformations.

Definition (matrix multiplication)
If is an matrix and is an matrix, then is defined to be the matrix for which for all .

This definition specifies the matrix product of two matrices, but it doesn't give us an algorithm for calculating it. Let's work that out in the context of a specific example:

Exercise (matrix product) Suppose that and . Consider the matrix defined so that, for all , the th column of is defined to be the product of and the th column of . Show that according to the definition of matrix multiplication.

Solution. Let be an arbitrary vector in . By definition,

Let's compute the expression on the right-hand side. Firstly, we have

Then, by linearity, we have

This demonstrates that is equal to for the matrix described in the problem.

The principle worked out in this exercise is general: the $k$th column of is the product of and the $k$th column of , for each column index . In other words,

where the notation means that are the columns of . We call this observation the product column rule.

## The invertible matrix theorem

The range or null space of a matrix is defined to be the range or null space of the corresponding linear transformation .

Exercise
Show that a matrix represents an injective transformation if and only if its null space is .

Solution. A linear transformation always maps a zero vector to the zero vector, so an linear transformation cannot map any other vector to the zero vector. Therefore, the null space of an injective transformation is the set containing only the zero vector.

If a transformation is not injective, then there are two distinct vectors and which map to the same vector . By linearity, the transformation maps to . Since is not equal to the vector, this implies that the null space contains more than just the zero vector. It follows that a matrix whose null space contains only the zero vector is indeed .

The rank of is defined to be the dimension of its range.

Example
The matrix has rank , because the span of its columns is all of . The null space has dimension 1, since the solution of is the span of .

For a matrix which is square (meaning that it represents a transformation from some space to itself), injectivity, surjectivity, and bijectivity are all equivalent:

Theorem (Invertible matrix theorem)
Suppose that is an matrix. Then the following are equivalent (that is, for a given matrix they are either all true or all false).

1. The transformation from to is bijective.
2. The range of is .
3. The null space of is .

In other words, for a linear transformation from to , bijectivity, surjectivity, and injectivity are equivalent.

Proof. We begin by showing that (ii) and (iii) are equivalent. If the columns of are linearly dependent, then the range of is spanned by fewer than vectors. Therefore, if the rank of is equal to , then the columns of are linearly independent. This implies that a linear combination of the columns is equal to the zero vector only if the weights are all zero. In other words, the only solution of the equation is the zero vector. In other words, the null space of is .

Conversely, if the null space of is , then the columns of are linearly , and the rank of is therefore equal to .

By definition of bijectivity, (ii) and (iii) together imply (i), and (i) implies (ii) and (iii). Therefore, the three statements are equivalent.

## The inverse matrix

If is invertible, then the inverse function is also a linear transformation:

Exercise
Show that if is a bijective linear transformation, then the inverse function is also linear.

Solution. Consider the linearity equation and two vectors and in the range of . Substitute and into the linearity equation for to obtain

which implies that

If is invertible, then matrix of the inverse of is called the inverse of and is denoted . The matrices and satisfy the equations , where denotes the identity matrix, which has ones along the diagonal starting at the top left entry and zeros elsewhere.

Example
If and then

Therefore for all So

Exercise
Let be a linear transformation defined to be a reflection across the -axis followed by a -degree clockwise rotation about the origin. Which of the following maps back to

Reflection across the -axis followed by a -degree counterclockwise rotation about the origin.
A -degree counterclockwise rotation about the origin followed by a reflection across the -axis.

Use the above example to write in terms of and , when and are invertible matrices.

Solution. The correct answer is (b). The transformation in (a) maps to

The example above illustrates geometrically that to invert a transformation represented by we may multiply the inverses of the two matrices and reverse the order: This is a general fact about function inverses and composition. We can see this algebraically by writing .

Exercise

Let be a non-zero matrix whose rank is .

• If and explain why there exists only one vector such that
• Suppose and show that there are vectors in for which the equation has no solution.
• If and both satisfy and for some fixed vector describe geometrically the set of points such that

Based on the above observations, can the equation have exactly solutions?

Solution.

1. If then the columns of form a basis for and so the range of is Therefore the corresponding linear transformation is invertible and the only vector that satisfies is given by

2. By definition, if the range of is not all of , then there exists a vector in which is not in the range of . In other words, there exists such that has no solution.

3. From

we see that the set of valid pairs is the diagonal line in

From and , we see that the equation can have or no solution. From , we see that if there are at least two distinct solutions, then there are in fact infinitely many solutions. So is not a possibility.

Bruno