Matrix icon

Matrix Math

Inverting a Matrix

As mentioned in the first section, to divide a matrix by another matrix, you must first invert the second matrix. We will compute it using the minors, cofactors, and adjugate method. We will invert the matrix:

A =
1 0 1
1 1 0
2 0 1
,  A-1 = adjA / detA

The inverse is obtained by dividing the adjugate by the determinant of the original matrix. The adjugate is the transpose of the matrix of cofactors. "Transpose" means to swap the rows and columns. The matrix of cofactors is the matrix of determinants obtained by crossing out each row and column just like with calculating the determinant, and then multiplying each determinant by the appropriate sign.

det(
1 0
1 1
) = 1 det(
1 1
2 0
) = -2And so on for all 9 elements...
minors(A) =
1 1 -2
0 -1 0
-1 -1 1
Now let's negate the odd-numbered elements...
cof(A) =
1 -1 -2
0 -1 0
-1 1 1

To obtain the inverse we must transpose to get the adjugate, and then divide by the determinant.

adj(A) =
1 0 -1
-1 -1 1
-2 0 1
det(
1 0 1
1 1 0
2 0 1
) = 1 - 0 + (-2) = -1
A-1 = (1 / -1) *
1 0 -1
-1 -1 1
-2 0 1
=
-1 0 1
1 1 -1
2 0 -1

Let's try multiplying a matrix by A and then by the inverse to see if we correctly inverted it.

25 11 -13
1 1 -1
-7 -2 4
*
1 0 1
1 1 0
2 0 1
=
10 11 12
0 1 0
-1 -2 -3
Multiplying by A...
10 11 12
0 1 0
-1 -2 -3
*
-1 0 1
1 1 -1
2 0 -1
=
25 11 -13
1 1 -1
-7 -2 4
The inverse correctly reverses the multiplication.

This can be used to solve equation systems where the number of equations is equal to the number of variables. Since a system can be represented as A * X = B, we can solve for X by multiplying both sides on the left by the inverse of A, where A is the coefficient matrix, X is the vector of unknowns, and B is the vector of constants. (A vector is a matrix with only one column.)

There are other methods to compute the inverse of a matrix, but this is a good one to start with. A more efficient method is to use Gaussian elimination to convert the matrix to 'row echelon form', while performing the same operations on an identity matrix, which will result in the inverse. This method will be explained in the echelon form section.

Multiplication Practice

Here's some multiplication exercises to try.

By the way, while it is a good idea to do these calculations by hand to learn the process, you should also check your work with a calculator or computer program. The calculator I use is called matrixcalc.org. It has lots of features, including a matrix inverse calculator. You can also install it to your desktop (some browsers may not support PWA installation).

If you can do these exercises by hand without a calculator, you shouldn't have any trouble with matrix multiplication.

Choose an exercise

1 2 3
4 5 6
7 8 9
*
0 1 0
0 0 1
1 0 0
9 8 7 6 5
4 3 2 1 0
*
1
2
3
4
5
-10 10
0 -10
*
1 2
3 4
1 2 3
4 5 6
7 8 9
*
1 2 3
4 5 6
7 8 9
1 2 3
4 5 6
7 8 9
*
0 0 1
0 1 0
1 0 0
1 -1 1
-1 1 -1
1 -1 1
*
-1 1 -1
1 -1 1
-1 1 -1

Matrix Inversion Practice

Let's invert a matrix. This will be a multi-stage process.

  1. First, we will need the determinant. The generated matrix will always be invertible, but if the determinant were 0 that would mean it isn't invertible.
  2. Then, transpose the matrix. (AT)
  3. The adjugate matrix is the matrix of cofactors of the transpose. (Adj(A) = Cof(AT))
  4. Done! The inverse is (1/det) * Adj(A).

Introduction to Matrices

Basic Matrix Operations

Matrices are an essential mathematical concept used in fields such as computer graphics, engineering and physics. A matrix is a rectangular array of elements, arranged in rows and columns. However, matrices are not just compact ways of storing data, as there are various mathematical operations that can be performed on them but not on plain numbers.

In this section, we will learn how to perform basic operations on matrices: addition, subtraction, multiplication. We will also learn how to calculate the determinant of a matrix, which is a very useful operation in many cases, as well as elementary row/column operations.

Addition and scalar multiplication

The standard arithmetic operations of addition, subtraction, multiplication and division do not completely apply to matrices. Addition and multiplication by a scalar (a normal number) are simple:

1 2
3 4
*2=
2 4
6 8
Multiplying a matrix by a scalar is easy...
1 2
3 4
+
1 0
0 1
=
2 2
3 5
Addition works as expected...

What about subtraction? Well, it's just addition with a negative number, so it's easy too. A - B = A + (-1 * B).

Matrix multiplication

Multiplication is weird:

1 2
3 4
*
1 0
0 1
=
1 2
3 4
This is clearly not just multiplying each element by the corresponding element...

Why is it like this? Well, defining multiplication like this turns out to allows all sorts of useful computations, such as solving systems of linear equations or graphics transformations. If we decided to just multiply normally, those operations wouldn't work. Matrix multiplication is defined as follows:

Let A be an m × n matrix and B be an n × p matrix. Then the result C is a matrix of size m × p, where each element Cij is the "dot product" of the ith row of A and the jth column of B, which means we take the sum of the products of pairs of elements one by one. For example, the number 4 from our example is obtained from the calculation 3 * 0 + 4 * 1 = 4. Notice that since both rows and columns have length n, it is guaranteed they will have equal length, so we can always do this. If the lengths don't match, there is no way to multiply the matrices. This means only square matrices can be raised to a power.

Fun fact: you don't need to do any calculation to solve our example, because we used the "identity matrix". The identity matrix is a square matrix with 1s on the diagonal and 0s everywhere else. It is the matrix equivalent of the number 1, and when you see it you know it doesn't change anything. It is also commutative, just like 1, so A * I = I * A = A. This is always true for normal numbers, but usually not for matrices.

We can see this operation is much more complicated than on the real numbers. It is not even defined for most pairs of matrices because we need the sizes to match, and in general it's not commutative. However, it is associative, which means that A * (B * C) = (A * B) * C.

... and division?

There is no 'direct' way to divide matrices. It is defined via multiplication by the inverse (just like how x / y = x * (1 / y)), but that is not always possible. Unlike numbers, non-zero matrices are not always invertible. In fact, they are only invertible if they are square and have a 'determinant' that is not 0. We will see how to find the inverse later. It involves dividing by the determinant, which is why we need it to be non-zero. In a sense, matrices with a determinant of 0 (also called singular matrices) lose information when multiplying with other matrices, so there is no way to reverse multiplying by them.

Finding the determinant

What is the determinant? Well, calculating the determinant is one of the common operations on matrices. There isn't really a good way to explain it in terms of normal numbers, so let's just see how it works on matrices.

det(
1 2
3 4
) = ?Determinants are a bit complicated... but for 2x2 matrices there is a simple formula.

2x2 matrices are a special case: we just do 1 * 4 - 2 * 3 = -2. We didn't get 0 so this matrix has an inverse. Now let's look at the general algorithm:

det(
1 2 3
4 5 6
7 8 9
) = ?I used a checkerboard pattern to make it easier to see the 'sign' of each element.Fun fact: there actually is a formula for this too. Look it up if you want, but we'll showcase the general algorithm.

This is how we do it:

  1. Pick a row or a column. Ones with zeroes are easier to calculate, but it always gives the same result.
  2. Start iterating over the elements in that row or column. For each element:
  3. Find the determinant of the matrix obtained by removing the row and column of the element. On paper, you can just imagine crossing out the row and column centered on your element. If only one element is non-zero you can literally do that, the others won't matter. We will see why in the next step.
  4. Multiply each determinant by the element and it's "sign". If the element is 0 don't bother calculating the determinant. The "sign" is 1 if the sum of the coordinates is even, else it's -1. It works like a chessboard where the top-left corner of the matrix has sign 1.
  5. Now simply add up the list of values you got. That is the determinant.

Do the calculation yourself to check that it works! For our matrix, picking the top row, the result will be 1 * (5 * 9 - 6 * 8) - 2 * (4 * 9 - 6 * 7) + 3 * (4 * 8 - 5 * 7) = 0. So, our matrix is not invertible.

In this case we didn't have too many calculations to do, but for larger matrices it can get quite complicated. In a 4x4 matrix, you'd have to calculate 4 determinants of 3x3 matrices, and each of those would have 3 determinants of 2x2 matrices. The calculations increase exponentially, but determinants larger than 4x4 are almost always calculated using a computer, via efficient algorithms. This is also why getting zeroes is important - you avoid sub-determinants.

Elementary row/column operations

We can do some operations on matrices that don't change their determinant. These operations are called "elementary row/column operations". They are very useful for dealing with matrices, and we will use them a lot. They are:

Let's use them to find the determinant of our 3x3 matrix in another way.

det(
1 2 3
4 5 6
7 8 9
) = det (
1 2 3
0 -3 -6
0 -6 -12
) = det (
-3 -6
-6 -12
) =-3 * -12 - -6 * -6 =36 - 36 = 0This is often easier than the general algorithm.

Notice how after the operations we got two rows that are just multiples of each other. When you see this, you can immediately say the determinant is 0, because you could obtain a zero row by multiplying the other row by a scalar and adding it to the first row. The same goes for columns.

In fact, it is always true that if the determinant is zero, you can obtain a zero row/column by elementary row/column operations. We say this means that the row/column is linearly dependent on the other rows/columns. Similarly, if the determinant is not zero, the rows/columns are linearly independent. This is a very important concept in linear equations and vector spaces, but we won't go into that here.

Be sure to check the multiplication exercises section to practice multiplication!