Lecture Notes on Vector Spaces and Subspaces

Author

Your Name

Published

February 3, 2025

Introduction

In this lecture, we will embark on an exploration of linear algebra, focusing on the fundamental concepts of vector spaces and subspaces. These abstract mathematical structures are essential tools in various fields, including computer science, physics, and engineering. We will begin by defining vector spaces and their axioms, then delve into subspaces, linear combinations, span, linear dependence, linear independence, basis, and dimension. Finally, we will touch upon the concept of rank and the Rank-Nullity Theorem. Understanding these concepts is crucial for building a solid foundation in linear algebra and its applications.

Vector Spaces

Definition

Definition 1 (Vector Space). A vector space $\mathcal{V}$ is a set of objects, called vectors, on which two operations are defined:

Vector Addition: For any two vectors $\mathbf{u}, \mathbf{v} \in \mathcal{V}$, there is a unique vector $\mathbf{u} + \mathbf{v} \in \mathcal{V}$ called the sum of $\mathbf{u}$ and $\mathbf{v}$.
Scalar Multiplication: For any scalar $c \in \mathbb{R}$ and any vector $\mathbf{u} \in \mathcal{V}$, there is a unique vector $c\mathbf{u} \in \mathcal{V}$ called the scalar product of $c$ and $\mathbf{u}$.

For $\mathcal{V}$ to be a vector space, these operations must satisfy the following eight axioms.

Axioms of Vector Spaces

The vector addition and scalar multiplication operations must adhere to specific axioms to qualify a set as a vector space. These axioms ensure the algebraic structure is consistent and well-behaved.

Axioms for Vector Addition

For all vectors $\mathbf{u}, \mathbf{v}, \mathbf{w} \in \mathcal{V}$:

Closure under Addition: If $\mathbf{u} \in \mathcal{V}$ and $\mathbf{v} \in \mathcal{V}$, then $\mathbf{u} + \mathbf{v} \in \mathcal{V}$.
Commutativity of Addition: $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$.
Associativity of Addition: $(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$.
Existence of a Zero Vector: There exists a vector $\mathbf{0} \in \mathcal{V}$ such that for all $\mathbf{u} \in \mathcal{V}$, $\mathbf{u} + \mathbf{0} = \mathbf{u}$.
Existence of Additive Inverses: For every $\mathbf{u} \in \mathcal{V}$, there exists a vector $-\mathbf{u} \in \mathcal{V}$ such that $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$.

Axioms for Scalar Multiplication

For all vectors $\mathbf{u}, \mathbf{v} \in \mathcal{V}$ and scalars $c, d \in \mathbb{R}$:

Closure under Scalar Multiplication: If $\mathbf{u} \in \mathcal{V}$ and $c \in \mathbb{R}$, then $c\mathbf{u} \in \mathcal{V}$.
Distributivity over Vector Addition: $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$.
Distributivity over Scalar Addition: $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$.
Scalar Multiplication Identity: $1\mathbf{u} = \mathbf{u}$.

These eight axioms are fundamental to the definition of a vector space and must be satisfied for any set to be considered a vector space under the defined operations.

Subspaces

Definition of a Subspace

Definition 2 (Subspace). A subspace $H$ of a vector space $\mathcal{V}$ is a subset of $\mathcal{V}$ that is itself a vector space under the same operations of vector addition and scalar multiplication defined on $\mathcal{V}$.

The Subspace Test

To determine if a subset $H$ of a vector space $\mathcal{V}$ is indeed a subspace, we apply the Subspace Test, which streamlines the verification process by focusing on the essential vector space properties.

Theorem 1 (Subspace Test). A subset $H$ of a vector space $\mathcal{V}$ is a subspace of $\mathcal{V}$ if and only if the following three conditions are met:

Zero Vector: The zero vector of $\mathcal{V}$ is in $H$, i.e., $\mathbf{0} \in H$.
Closure under Addition: For any vectors $\mathbf{u}, \mathbf{v} \in H$, their sum $\mathbf{u} + \mathbf{v}$ is also in $H$.
Closure under Scalar Multiplication: For any vector $\mathbf{u} \in H$ and any scalar $c \in \mathbb{R}$, the scalar product $c\mathbf{u}$ is also in $H$.

Explanation of Subspace Test Conditions

The Subspace Test simplifies checking if a subset is a subspace by verifying only three critical properties.

Zero Vector in Subspace

The first condition, requiring the zero vector to be in $H$, is essential because every vector space must contain a zero vector as its additive identity. If $H$ does not contain $\mathbf{0}$, it cannot satisfy the axioms of a vector space and thus cannot be a subspace.

Closure under Vector Addition

The second condition, closure under vector addition, ensures that the sum of any two vectors in $H$ remains within $H$. This is necessary for vector addition to be a valid operation within $H$, consistent with the definition of a vector space.

Closure under Scalar Multiplication

The third condition, closure under scalar multiplication, ensures that multiplying any vector in $H$ by a scalar results in a vector that is still in $H$. This is necessary for scalar multiplication to be a valid operation within $H$, again consistent with the definition of a vector space.

Examples of Subspaces and Non-Subspaces

To illustrate the concept of subspaces, let’s examine some examples within the vector space $\mathbb{R}^2$.

Trivial Subspaces

Example 1 (Trivial Subspaces). Every vector space $\mathcal{V}$ has at least two subspaces, known as trivial subspaces:

The Zero Subspace: $H = \{\mathbf{0}\}$, which contains only the zero vector. This trivially satisfies all subspace conditions.
The Vector Space Itself: $H = \mathcal{V}$. Since a vector space is a subset of itself and satisfies all vector space axioms, $\mathcal{V}$ is a subspace of $\mathcal{V}$.

The x-axis in $\mathbb{R}^2$ is a Subspace

Example 2 (The x-axis in $\mathbb{R}^2$). Let $H = \{(x, 0) \in \mathbb{R}^2 \mid x \in \mathbb{R}\}$ be the set of all vectors on the x-axis in $\mathbb{R}^2$. We verify if $H$ is a subspace of $\mathbb{R}^2$ using the Subspace Test:

Zero Vector: The zero vector $\mathbf{0} = (0, 0)$ is in $H$ because it has a y-coordinate of 0 (when $x=0$).
Closure under Addition: Let $\mathbf{u} = (x_1, 0) \in H$ and $\mathbf{v} = (x_2, 0) \in H$. Then $\mathbf{u} + \mathbf{v} = (x_1 + x_2, 0)$. Since $x_1 + x_2$ is a real number, $\mathbf{u} + \mathbf{v}$ is in $H$.
Closure under Scalar Multiplication: Let $\mathbf{u} = (x, 0) \in H$ and $c \in \mathbb{R}$. Then $c\mathbf{u} = c(x, 0) = (cx, 0)$. Since $cx$ is a real number, $c\mathbf{u}$ is in $H$.

Since $H$ satisfies all three conditions, $H$ is a subspace of $\mathbb{R}^2$.

A Line Not Through the Origin in $\mathbb{R}^2$ is Not a Subspace

Example 3 (A Line Not Through the Origin in $\mathbb{R}^2$). Let $K = \{(x, 1) \in \mathbb{R}^2 \mid x \in \mathbb{R}\}$ be the set of all vectors on the line $y=1$ in $\mathbb{R}^2$. We check if $K$ is a subspace of $\mathbb{R}^2$:

Zero Vector: The zero vector $\mathbf{0} = (0, 0)$ is not in $K$ because vectors in $K$ must have a y-coordinate of 1.

Since $K$ fails to contain the zero vector, the first condition of the Subspace Test is not satisfied. Therefore, $K$ is not a subspace of $\mathbb{R}^2$. It is unnecessary to check the other conditions once one condition fails.

These examples illustrate how to apply the Subspace Test to determine whether a given subset of a vector space is indeed a subspace. Understanding these fundamental concepts is crucial for further exploration in linear algebra.

Linear Combinations and Span

Linear Combinations

Definition 3 (Linear Combination). Given a set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$, a linear combination of these vectors is any vector $\mathbf{y}$ that can be expressed in the form: \[\mathbf{y} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k\] where $c_1, c_2, \dots, c_k$ are scalars.

In essence, a linear combination is formed by multiplying each vector in the set by a scalar and summing the results. The scalars can be any real numbers, allowing for a wide range of combinations.

Span of a Set of Vectors

Definition 4 (Span). The span of a set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$, denoted as $\text{span}(S)$ or $\text{span}\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$, is the set of all possible linear combinations of $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$. That is: \[\text{span}(S) = \{c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k \mid c_1, c_2, \dots, c_k \in \mathbb{R} \}\]

The span of a set of vectors effectively describes the "space" that can be "reached" or "generated" by taking all possible linear combinations of those vectors. It is a fundamental concept for understanding the structure of vector spaces and subspaces.

The Span of a Set is a Subspace

A crucial property of the span is that it always forms a subspace of the original vector space. This is formalized in the following theorem:

Theorem 2 (Span is a Subspace). For any set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$, the span of $S$, denoted as $\text{span}(S)$, is a subspace of $\mathcal{V}$.

This theorem is significant because it provides a method for constructing subspaces. Given any set of vectors, their span will always be a subspace.

Examples of Span in $\mathbb{R}^2$

Example 4 (Span in $\mathbb{R}^2$). Consider the vector space $\mathbb{R}^2$.

These examples in $\mathbb{R}^2$ provide a geometric intuition for the concept of span, illustrating how different sets of vectors can generate different subspaces.

Linear Dependence and Linear Independence

Linear dependence and linear independence are crucial concepts in linear algebra that describe the relationships between vectors in a set. They determine whether vectors in a set are redundant or essential for spanning a space.

Linear Dependence

Definition 5 (Linear Dependence). A set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$ is linearly dependent if there exist scalars $c_1, c_2, \dots, c_k$, not all zero, such that: \[c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k = \mathbf{0}\] This equation is called a linear dependence relation among $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$. A linear combination where at least one scalar is non-zero and the combination equals the zero vector is termed a non-trivial linear combination.

Linear dependence implies that at least one vector in the set can be expressed as a linear combination of the others. In other words, there is redundancy within the set of vectors.

Linear Independence

Definition 6 (Linear Independence). A set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$ is linearly independent if the only scalars $c_1, c_2, \dots, c_k$ that satisfy the equation: \[c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k = \mathbf{0}\] are $c_1 = c_2 = \dots = c_k = 0$. In this case, the only linear combination that equals the zero vector is the trivial linear combination, where all scalars are zero.

Linear independence means that none of the vectors in the set can be written as a linear combination of the others. Each vector in a linearly independent set is "essential" and contributes uniquely to the span of the set.

Examples in $\mathbb{R}^2$

To illustrate linear dependence and independence, consider vectors in $\mathbb{R}^2$.

Linear Dependence Example

Example 5 (Linear Dependence in $\mathbb{R}^2$). Let $\mathbf{v}_1 = (1, 2)$ and $\mathbf{v}_2 = (2, 4)$ in $\mathbb{R}^2$. To check for linear dependence, we consider the equation $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 = \mathbf{0}$: \[c_1(1, 2) + c_2(2, 4) = (0, 0)\] This vector equation is equivalent to the system of linear equations: \[\begin{aligned} c_1 + 2c_2 &= 0 \\ 2c_1 + 4c_2 &= 0 \end{aligned}\] We observe that $\mathbf{v}_2 = 2\mathbf{v}_1$. Thus, we can choose $c_1 = 2$ and $c_2 = -1$ (not both zero) such that $2\mathbf{v}_1 - \mathbf{v}_2 = \mathbf{0}$. Therefore, $\{\mathbf{v}_1, \mathbf{v}_2\}$ is linearly dependent.

Linear Independence Example

Example 6 (Linear Independence in $\mathbb{R}^2$). Let $\mathbf{v}_1 = (1, 0)$ and $\mathbf{v}_2 = (0, 1)$ in $\mathbb{R}^2$. Consider the equation $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 = \mathbf{0}$: \[c_1(1, 0) + c_2(0, 1) = (0, 0)\] This simplifies to $(c_1, c_2) = (0, 0)$, which implies $c_1 = 0$ and $c_2 = 0$. The only solution is the trivial solution. Therefore, $\{\mathbf{v}_1, \mathbf{v}_2\}$ is linearly independent.

These examples demonstrate how to determine whether a set of vectors is linearly dependent or linearly independent by examining the solutions to the homogeneous vector equation.

Basis and Dimension

Basis and dimension are fundamental concepts that provide a way to characterize and quantify vector spaces. A basis provides a minimal set of vectors that can generate the entire vector space, while the dimension quantifies the "size" of the vector space.

Definition of a Basis

Definition 7 (Basis). A basis for a vector space $\mathcal{V}$ is a set of vectors $\mathcal{B} = \{\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_n\}$ in $\mathcal{V}$ that satisfies two essential conditions:

Linear Independence: The set $\mathcal{B}$ is linearly independent. This means that no vector in $\mathcal{B}$ can be expressed as a linear combination of the other vectors in $\mathcal{B}$.
Spanning Set: The set $\mathcal{B}$ spans $\mathcal{V}$. This means that every vector in $\mathcal{V}$ can be expressed as a linear combination of the vectors in $\mathcal{B}$. In other words, $\text{span}(\mathcal{B}) = \mathcal{V}$.

A basis can be thought of as a minimal "scaffolding" for a vector space. It contains just enough vectors to construct any vector in the space through linear combinations, without any redundancy.

Finite-Dimensional and Infinite-Dimensional Vector Spaces

Vector spaces can be classified based on whether they have a basis with a finite number of vectors.

Definition 8 (Finite-Dimensional Vector Space). A vector space $\mathcal{V}$ is called finite-dimensional if it possesses a basis consisting of a finite number of vectors. If a vector space does not have a finite basis, it is called infinite-dimensional.

In this lecture, we primarily focus on finite-dimensional vector spaces, which are commonly encountered in many areas of applied mathematics and computer science.

Dimension of a Vector Space

For finite-dimensional vector spaces, the dimension is a fundamental property that quantifies the number of vectors in any basis for that space.

Definition 9 (Dimension). The dimension of a finite-dimensional vector space $\mathcal{V}$, denoted by $\text{dim}(\mathcal{V})$, is the number of vectors in any basis for $\mathcal{V}$. A crucial theorem in linear algebra states that all bases for a finite-dimensional vector space contain the same number of vectors. Thus, the dimension is a well-defined property of the vector space itself, indicating its "size" or degrees of freedom.

The dimension provides essential information about the structure of a vector space. For example, in geometric terms, the dimension corresponds to the number of independent directions within the space.

Examples of Bases and Dimensions

Let’s examine the standard bases and dimensions for common vector spaces $\mathbb{R}^2$, $\mathbb{R}^3$, and generalize to $\mathbb{R}^n$.

Basis and Dimension of $\mathbb{R}^2$

Example 7 (Basis and Dimension of $\mathbb{R}^2$). The standard basis for $\mathbb{R}^2$ is the set $\mathcal{B} = \{\mathbf{e}_1, \mathbf{e}_2\} = \{(1, 0), (0, 1)\}$.

Linear Independence: $\{\mathbf{e}_1, \mathbf{e}_2\}$ is linearly independent, as shown in [example:Linear Independence in $\mathbb{R}^2$].
Spanning Set: $\{\mathbf{e}_1, \mathbf{e}_2\}$ spans $\mathbb{R}^2$. Any vector $\mathbf{v} = (x, y) \in \mathbb{R}^2$ can be written as $\mathbf{v} = x\mathbf{e}_1 + y\mathbf{e}_2 = x(1, 0) + y(0, 1) = (x, 0) + (0, y) = (x, y)$.

Since $\mathcal{B}$ is linearly independent and spans $\mathbb{R}^2$, it is a basis for $\mathbb{R}^2$. The dimension of $\mathbb{R}^2$ is the number of vectors in this basis, which is 2. Thus, $\text{dim}(\mathbb{R}^2) = 2$.

Basis and Dimension of $\mathbb{R}^3$

Example 8 (Basis and Dimension of $\mathbb{R}^3$). The standard basis for $\mathbb{R}^3$ is $\mathcal{B} = \{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\} = \{(1, 0, 0), (0, 1, 0), (0, 0, 1)\}$.

Linear Independence: It can be shown that $\{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\}$ is linearly independent.
Spanning Set: $\{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\}$ spans $\mathbb{R}^3$. Any vector $\mathbf{v} = (x, y, z) \in \mathbb{R}^3$ can be written as $\mathbf{v} = x\mathbf{e}_1 + y\mathbf{e}_2 + z\mathbf{e}_3$.

Thus, $\mathcal{B}$ is a basis for $\mathbb{R}^3$, and the dimension of $\mathbb{R}^3$ is 3, i.e., $\text{dim}(\mathbb{R}^3) = 3$.

Basis and Dimension of $\mathbb{R}^n$

Example 9 (Basis and Dimension of $\mathbb{R}^n$). Generalizing, the standard basis for $\mathbb{R}^n$ is $\mathcal{B} = \{\mathbf{e}_1, \mathbf{e}_2, \dots, \mathbf{e}_n\}$, where $\mathbf{e}_i$ is the vector with a 1 in the $i$-th position and 0 elsewhere.

Linear Independence: The standard basis for $\mathbb{R}^n$ is linearly independent.
Spanning Set: The standard basis for $\mathbb{R}^n$ spans $\mathbb{R}^n$. Any vector $\mathbf{v} = (x_1, x_2, \dots, x_n) \in \mathbb{R}^n$ can be written as $\mathbf{v} = x_1\mathbf{e}_1 + x_2\mathbf{e}_2 + \dots + x_n\mathbf{e}_n$.

Therefore, $\mathcal{B}$ is a basis for $\mathbb{R}^n$, and the dimension of $\mathbb{R}^n$ is $n$, i.e., $\text{dim}(\mathbb{R}^n) = n$.

Finding a Basis using Row Reduction

Row reduction is a practical method to find a basis for the column space of a matrix, which is the span of its column vectors. This is particularly useful when we are given a set of vectors that span a subspace and we need to find a basis for that subspace.

Procedure for Finding a Basis using Row Reduction

The procedure involves transforming a matrix into row-echelon form and identifying the pivot columns.

Form a Matrix: Construct a matrix $A$ where the given vectors are the columns of $A$.
Row Reduce to Echelon Form: Use elementary row operations to transformmatrix $A$ into row-echelon form, say matrix $R$.
Identify Pivot Columns: Determine the pivot columns in the row-echelon form $R$. Pivot columns are the columns that contain a leading entry (pivot).
Basis Vectors: The columns in the original matrix $A$ that correspond to the pivot columns in the row-echelon form $R$ form a basis for the column space of $A$, $\text{Col}(A) = \text{span}\{\text{columns of } A\}$.

This method efficiently extracts a linearly independent subset from the original set of vectors that still spans the same column space, thus providing a basis. An example illustrating this process is shown below.

::: {.example:Row Reduction Basis .example} Example 10 (Finding a Basis using Row Reduction). Let’s find a basis for the subspace spanned by the vectors $\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$, $\mathbf{v}_2 = \begin{pmatrix} 2 \\ 4 \\ 6 \end{pmatrix}$, $\mathbf{v}_3 = \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix}$, and $\mathbf{v}_4 = \begin{pmatrix} 2 \\ 5 \\ 8 \end{pmatrix}$.

Step 1: Form a Matrix Construct a matrix $A$ with these vectors as columns: \[A = \begin{pmatrix} 1 & 2 & 1 & 2 \\ 2 & 4 & 3 & 5 \\ 3 & 6 & 5 & 8 \end{pmatrix}\]

Step 2: Row Reduce to Echelon Form Perform row operations to get the row-echelon form $R$: \[\begin{aligned} \begin{pmatrix} 1 & 2 & 1 & 2 \\ 2 & 4 & 3 & 5 \\ 3 & 6 & 5 & 8 \end{pmatrix} &\xrightarrow{R_2 \leftarrow R_2 - 2R_1, R_3 \leftarrow R_3 - 3R_1} \begin{pmatrix} 1 & 2 & 1 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 2 & 2 \end{pmatrix} \\ &\xrightarrow{R_3 \leftarrow R_3 - 2R_2} \begin{pmatrix} 1 & 2 & 1 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{pmatrix} = R \end{aligned}\] The row-echelon form is $R = \begin{pmatrix} 1 & 2 & 1 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{pmatrix}$.

Step 3: Identify Pivot Columns The pivot columns in $R$ are the first and the third columns (containing the leading entries 1 and 1).

Step 4: Basis Vectors The corresponding columns in the original matrix $A$ are the first and the third columns, $\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$ and $\mathbf{v}_3 = \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix}$.

Thus, a basis for the subspace spanned by $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3, \mathbf{v}_4\}$ is $\{\mathbf{v}_1, \mathbf{v}_3\} = \left\{ \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}, \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix} \right\}$.

::::

Rank and the Rank-Nullity Theorem

The concept of rank provides a measure of the "size" of the image or column space of a matrix, indicating the number of linearly independent columns (or rows). The Rank-Nullity Theorem is a fundamental result that connects the rank of a matrix to the dimension of its null space, offering a deep insight into the structure of linear transformations.

Rank of a Matrix

Definition 10 (Rank of a Matrix). The rank of a matrix $A$, denoted as $\text{rank}(A)$, is defined as the dimension of the column space of $A$. Equivalently, it is also the dimension of the row space of $A$. The rank of a matrix represents the maximum number of linearly independent columns (or rows) in the matrix.

The rank of a matrix is a non-negative integer that provides crucial information about the matrix’s properties and the linear transformation it represents. A higher rank indicates a "larger" column space and, consequently, a "more significant" linear transformation in terms of dimensionality.

The Rank-Nullity Theorem

Theorem 3 (Rank-Nullity Theorem). Let $A$ be an $m \times n$ matrix. Then the sum of the dimension of the null space of $A$ (also called the nullity of $A$) and the rank of $A$ is equal to the number of columns of $A$, which is $n$. In equation form: \[\text{dim}(\text{Nul}(A)) + \text{rank}(A) = n\] Equivalently, for a linear transformation $\Phi: \mathcal{V}\to \mathcal{W}$ represented by matrix $A$, where $\mathcal{V}$ is $n$-dimensional, the theorem states: \[\text{dim}(\text{ker}(\Phi)) + \text{dim}(\text{Im}(\Phi)) = \text{dim}(\mathcal{V})\] where $\text{Nul}(A) = \text{ker}(\Phi)$ is the kernel (null space) of $A$ (or $\Phi$), and $\text{rank}(A) = \text{dim}(\text{Col}(A)) = \text{dim}(\text{Im}(\Phi))$ is the rank of $A$ (or the dimension of the image of $\Phi$).

The Rank-Nullity Theorem describes the relationship between the dimensions of the null space and the column space of a matrix, stating that their sum equals the number of columns of the matrix.

Interpretation and Significance

The Rank-Nullity Theorem reveals a fundamental trade-off: for a given matrix or linear transformation, the higher the rank (i.e., the larger the image), the smaller the nullity (i.e., the smaller the kernel), and vice versa, with their dimensions always summing up to the dimension of the domain.

Dimension of Solution Space: $\text{dim}(\text{Nul}(A))$ represents the number of free variables in the solution to the homogeneous equation $A\mathbf{x} = \mathbf{0}$. It quantifies the "size" of the solution set.
Dimension of Image: $\text{rank}(A) = \text{dim}(\text{Col}(A))$ represents the dimension of the space spanned by the columns of $A$, which is the image of the linear transformation. It quantifies the "reach" of the transformation.
Domain Dimension: $n = \text{dim}(\mathcal{V})$ is the dimension of the input space.

The Rank-Nullity Theorem is a powerful tool in linear algebra, providing a way to understand the relationship between the input and output spaces of a linear transformation and the structure of the solutions to linear systems. It is used in various applications, including determining the existence and uniqueness of solutions to linear equations and understanding the properties of linear transformations.

Conclusion

In this lecture, we have established the foundational concepts of vector spaces and subspaces, which are crucial for linear algebra. We began by defining vector spaces through their axioms, ensuring a rigorous understanding of their structure. We then explored subspaces, learning how to verify if a subset qualifies as a subspace using the Subspace Test. Key operations of linear combinations and span were introduced, demonstrating how sets of vectors can generate subspaces. We differentiated between linear dependence and independence, concepts essential for understanding the redundancy and efficiency of vector sets. Furthermore, we defined basis and dimension, providing tools to characterize the size and structure of vector spaces, and touched upon the method of row reduction for finding a basis. Finally, we introduced the rank of a matrix and the Rank-Nullity Theorem, which unveils a fundamental relationship between the null space and column space of a matrix.

Key Takeaways

Vector Spaces: Defined by axioms that govern vector addition and scalar multiplication, providing a general framework for linear structures.
Subspaces: Subsets of vector spaces that are themselves vector spaces, inheriting operations and properties from the parent space, verifiable by the Subspace Test.
Linear Combinations and Span: Linear combinations create new vectors from existing sets, and the span of a vector set forms a subspace, representing all reachable vectors through linear combinations.
Linear Dependence and Independence: Linear dependence indicates redundancy within a vector set, while linear independence signifies that each vector is essential for spanning their space.
Basis and Dimension: A basis is a minimal, linearly independent spanning set for a vector space, and the dimension is the number of vectors in any basis, quantifying the space’s size.
Rank and Rank-Nullity Theorem: Rank measures the dimension of the column space, and the Rank-Nullity Theorem relates rank to the dimension of the null space, providing a fundamental insight into linear transformations.

Further Reading and Next Steps

To deepen your understanding and extend your knowledge, consider exploring the following topics:

Applications to Linear Systems: Investigate how vector space concepts are used to solve systems of linear equations, including existence and uniqueness of solutions.
Linear Transformations and Matrices: Study linear transformations in detail and how they are represented by matrices, including matrix operations and properties.
Practical Applications: Explore real-world applications of linear algebra in computer graphics, data analysis, machine learning, and engineering to appreciate the practical significance of these concepts.

Understanding these fundamental concepts is crucial for further studies in linear algebra and its applications across various scientific and engineering disciplines.

--- title: "Lecture Notes on Vector Spaces and Subspaces" author: "Your Name" date: "2025-02-03" format: html: toc: true # Table of Contents toc-depth: 2 code-tools: true theme: cosmo # Or "journal" for Distill-like minimalism --- # Introduction In this lecture, we will embark on an exploration of linear algebra, focusing on the fundamental concepts of vector spaces and subspaces. These abstract mathematical structures are essential tools in various fields, including computer science, physics, and engineering. We will begin by defining vector spaces and their axioms, then delve into subspaces, linear combinations, span, linear dependence, linear independence, basis, and dimension. Finally, we will touch upon the concept of rank and the Rank-Nullity Theorem. Understanding these concepts is crucial for building a solid foundation in linear algebra and its applications. # Vector Spaces ## Definition :::: tcolorbox ::: definition **Definition 1** (Vector Space). A **vector space** $\mathcal{V}$ is a set of objects, called **vectors**, on which two operations are defined: 1. **Vector Addition**: For any two vectors $\mathbf{u}, \mathbf{v} \in \mathcal{V}$, there is a unique vector $\mathbf{u} + \mathbf{v} \in \mathcal{V}$ called the **sum** of $\mathbf{u}$ and $\mathbf{v}$. 2. **Scalar Multiplication**: For any scalar $c \in \mathbb{R}$ and any vector $\mathbf{u} \in \mathcal{V}$, there is a unique vector $c\mathbf{u} \in \mathcal{V}$ called the **scalar product** of $c$ and $\mathbf{u}$. For $\mathcal{V}$ to be a vector space, these operations must satisfy the following eight axioms. ::: :::: ## Axioms of Vector Spaces The vector addition and scalar multiplication operations must adhere to specific axioms to qualify a set as a vector space. These axioms ensure the algebraic structure is consistent and well-behaved. ### Axioms for Vector Addition For all vectors $\mathbf{u}, \mathbf{v}, \mathbf{w} \in \mathcal{V}$: 1. **Closure under Addition**: If $\mathbf{u} \in \mathcal{V}$ and $\mathbf{v} \in \mathcal{V}$, then $\mathbf{u} + \mathbf{v} \in \mathcal{V}$. 2. **Commutativity of Addition**: $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$. 3. **Associativity of Addition**: $(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$. 4. **Existence of a Zero Vector**: There exists a vector $\mathbf{0} \in \mathcal{V}$ such that for all $\mathbf{u} \in \mathcal{V}$, $\mathbf{u} + \mathbf{0} = \mathbf{u}$. 5. **Existence of Additive Inverses**: For every $\mathbf{u} \in \mathcal{V}$, there exists a vector $-\mathbf{u} \in \mathcal{V}$ such that $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$. ### Axioms for Scalar Multiplication For all vectors $\mathbf{u}, \mathbf{v} \in \mathcal{V}$ and scalars $c, d \in \mathbb{R}$: 6. **Closure under Scalar Multiplication**: If $\mathbf{u} \in \mathcal{V}$ and $c \in \mathbb{R}$, then $c\mathbf{u} \in \mathcal{V}$. 7. **Distributivity over Vector Addition**: $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$. 8. **Distributivity over Scalar Addition**: $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$. 9. **Scalar Multiplication Identity**: $1\mathbf{u} = \mathbf{u}$. These eight axioms are fundamental to the definition of a vector space and must be satisfied for any set to be considered a vector space under the defined operations. # Subspaces ## Definition of a Subspace :::: tcolorbox ::: definition **Definition 2** (Subspace). A **subspace** $H$ of a vector space $\mathcal{V}$ is a subset of $\mathcal{V}$ that is itself a vector space under the same operations of vector addition and scalar multiplication defined on $\mathcal{V}$. ::: :::: ## The Subspace Test To determine if a subset $H$ of a vector space $\mathcal{V}$ is indeed a subspace, we apply the Subspace Test, which streamlines the verification process by focusing on the essential vector space properties. :::: tcolorbox ::: theorem **Theorem 1** (Subspace Test). *A subset $H$ of a vector space $\mathcal{V}$ is a subspace of $\mathcal{V}$ if and only if the following three conditions are met:* 1. ***Zero Vector**: The zero vector of $\mathcal{V}$ is in $H$, i.e., $\mathbf{0} \in H$.* 2. ***Closure under Addition**: For any vectors $\mathbf{u}, \mathbf{v} \in H$, their sum $\mathbf{u} + \mathbf{v}$ is also in $H$.* 3. ***Closure under Scalar Multiplication**: For any vector $\mathbf{u} \in H$ and any scalar $c \in \mathbb{R}$, the scalar product $c\mathbf{u}$ is also in $H$.* ::: :::: ## Explanation of Subspace Test Conditions The Subspace Test simplifies checking if a subset is a subspace by verifying only three critical properties. ### Zero Vector in Subspace The first condition, requiring the zero vector to be in $H$, is essential because every vector space must contain a zero vector as its additive identity. If $H$ does not contain $\mathbf{0}$, it cannot satisfy the axioms of a vector space and thus cannot be a subspace. ### Closure under Vector Addition The second condition, closure under vector addition, ensures that the sum of any two vectors in $H$ remains within $H$. This is necessary for vector addition to be a valid operation within $H$, consistent with the definition of a vector space. ### Closure under Scalar Multiplication The third condition, closure under scalar multiplication, ensures that multiplying any vector in $H$ by a scalar results in a vector that is still in $H$. This is necessary for scalar multiplication to be a valid operation within $H$, again consistent with the definition of a vector space. ## Examples of Subspaces and Non-Subspaces To illustrate the concept of subspaces, let's examine some examples within the vector space $\mathbb{R}^2$. ### Trivial Subspaces :::: tcolorbox ::: example **Example 1** (Trivial Subspaces). Every vector space $\mathcal{V}$ has at least two subspaces, known as **trivial subspaces**: - **The Zero Subspace**: $H = \{\mathbf{0}\}$, which contains only the zero vector. This trivially satisfies all subspace conditions. - **The Vector Space Itself**: $H = \mathcal{V}$. Since a vector space is a subset of itself and satisfies all vector space axioms, $\mathcal{V}$ is a subspace of $\mathcal{V}$. ::: :::: ### The x-axis in $\mathbb{R}^2$ is a Subspace :::: tcolorbox ::: example **Example 2** (The x-axis in $\mathbb{R}^2$). Let $H = \{(x, 0) \in \mathbb{R}^2 \mid x \in \mathbb{R}\}$ be the set of all vectors on the x-axis in $\mathbb{R}^2$. We verify if $H$ is a subspace of $\mathbb{R}^2$ using the Subspace Test: 1. **Zero Vector**: The zero vector $\mathbf{0} = (0, 0)$ is in $H$ because it has a y-coordinate of 0 (when $x=0$). 2. **Closure under Addition**: Let $\mathbf{u} = (x_1, 0) \in H$ and $\mathbf{v} = (x_2, 0) \in H$. Then $\mathbf{u} + \mathbf{v} = (x_1 + x_2, 0)$. Since $x_1 + x_2$ is a real number, $\mathbf{u} + \mathbf{v}$ is in $H$. 3. **Closure under Scalar Multiplication**: Let $\mathbf{u} = (x, 0) \in H$ and $c \in \mathbb{R}$. Then $c\mathbf{u} = c(x, 0) = (cx, 0)$. Since $cx$ is a real number, $c\mathbf{u}$ is in $H$. Since $H$ satisfies all three conditions, $H$ is a subspace of $\mathbb{R}^2$. ::: :::: ### A Line Not Through the Origin in $\mathbb{R}^2$ is Not a Subspace :::: tcolorbox ::: example **Example 3** (A Line Not Through the Origin in $\mathbb{R}^2$). Let $K = \{(x, 1) \in \mathbb{R}^2 \mid x \in \mathbb{R}\}$ be the set of all vectors on the line $y=1$ in $\mathbb{R}^2$. We check if $K$ is a subspace of $\mathbb{R}^2$: 1. **Zero Vector**: The zero vector $\mathbf{0} = (0, 0)$ is not in $K$ because vectors in $K$ must have a y-coordinate of 1. Since $K$ fails to contain the zero vector, the first condition of the Subspace Test is not satisfied. Therefore, $K$ is not a subspace of $\mathbb{R}^2$. It is unnecessary to check the other conditions once one condition fails. ::: :::: These examples illustrate how to apply the Subspace Test to determine whether a given subset of a vector space is indeed a subspace. Understanding these fundamental concepts is crucial for further exploration in linear algebra. # Linear Combinations and Span ## Linear Combinations :::: tcolorbox ::: definition **Definition 3** (Linear Combination). Given a set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$, a **linear combination** of these vectors is any vector $\mathbf{y}$ that can be expressed in the form: $$\mathbf{y} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k$$ where $c_1, c_2, \dots, c_k$ are scalars. ::: :::: In essence, a linear combination is formed by multiplying each vector in the set by a scalar and summing the results. The scalars can be any real numbers, allowing for a wide range of combinations. ## Span of a Set of Vectors :::: tcolorbox ::: definition **Definition 4** (Span). The **span** of a set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$, denoted as $\text{span}(S)$ or $\text{span}\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$, is the set of all possible linear combinations of $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$. That is: $$\text{span}(S) = \{c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k \mid c_1, c_2, \dots, c_k \in \mathbb{R} \}$$ ::: :::: The span of a set of vectors effectively describes the \"space\" that can be \"reached\" or \"generated\" by taking all possible linear combinations of those vectors. It is a fundamental concept for understanding the structure of vector spaces and subspaces. ## The Span of a Set is a Subspace A crucial property of the span is that it always forms a subspace of the original vector space. This is formalized in the following theorem: :::: tcolorbox ::: theorem **Theorem 2** (Span is a Subspace). *For any set of vectors $S = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$, the span of $S$, denoted as $\text{span}(S)$, is a subspace of $\mathcal{V}$.* ::: :::: This theorem is significant because it provides a method for constructing subspaces. Given any set of vectors, their span will always be a subspace. ## Examples of Span in $\mathbb{R}^2$ :::: tcolorbox ::: example **Example 4** (Span in $\mathbb{R}^2$). Consider the vector space $\mathbb{R}^2$. ::: :::: These examples in $\mathbb{R}^2$ provide a geometric intuition for the concept of span, illustrating how different sets of vectors can generate different subspaces. # Linear Dependence and Linear Independence Linear dependence and linear independence are crucial concepts in linear algebra that describe the relationships between vectors in a set. They determine whether vectors in a set are redundant or essential for spanning a space. ## Linear Dependence :::: tcolorbox ::: definition **Definition 5** (Linear Dependence). A set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$ is **linearly dependent** if there exist scalars $c_1, c_2, \dots, c_k$, *not all zero*, such that: $$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k = \mathbf{0}$$ This equation is called a **linear dependence relation** among $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$. A linear combination where at least one scalar is non-zero and the combination equals the zero vector is termed a **non-trivial linear combination**. ::: :::: Linear dependence implies that at least one vector in the set can be expressed as a linear combination of the others. In other words, there is redundancy within the set of vectors. ## Linear Independence :::: tcolorbox ::: definition **Definition 6** (Linear Independence). A set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$ in a vector space $\mathcal{V}$ is **linearly independent** if the only scalars $c_1, c_2, \dots, c_k$ that satisfy the equation: $$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k = \mathbf{0}$$ are $c_1 = c_2 = \dots = c_k = 0$. In this case, the only linear combination that equals the zero vector is the **trivial linear combination**, where all scalars are zero. ::: :::: Linear independence means that none of the vectors in the set can be written as a linear combination of the others. Each vector in a linearly independent set is \"essential\" and contributes uniquely to the span of the set. ## Examples in $\mathbb{R}^2$ To illustrate linear dependence and independence, consider vectors in $\mathbb{R}^2$. ### Linear Dependence Example :::: tcolorbox ::: example **Example 5** (Linear Dependence in $\mathbb{R}^2$). Let $\mathbf{v}_1 = (1, 2)$ and $\mathbf{v}_2 = (2, 4)$ in $\mathbb{R}^2$. To check for linear dependence, we consider the equation $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 = \mathbf{0}$: $$c_1(1, 2) + c_2(2, 4) = (0, 0)$$ This vector equation is equivalent to the system of linear equations: $$\begin{aligned} c_1 + 2c_2 &= 0 \\ 2c_1 + 4c_2 &= 0 \end{aligned}$$ We observe that $\mathbf{v}_2 = 2\mathbf{v}_1$. Thus, we can choose $c_1 = 2$ and $c_2 = -1$ (not both zero) such that $2\mathbf{v}_1 - \mathbf{v}_2 = \mathbf{0}$. Therefore, $\{\mathbf{v}_1, \mathbf{v}_2\}$ is linearly dependent. ::: :::: ### Linear Independence Example :::: tcolorbox ::: example **Example 6** (Linear Independence in $\mathbb{R}^2$). Let $\mathbf{v}_1 = (1, 0)$ and $\mathbf{v}_2 = (0, 1)$ in $\mathbb{R}^2$. Consider the equation $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 = \mathbf{0}$: $$c_1(1, 0) + c_2(0, 1) = (0, 0)$$ This simplifies to $(c_1, c_2) = (0, 0)$, which implies $c_1 = 0$ and $c_2 = 0$. The only solution is the trivial solution. Therefore, $\{\mathbf{v}_1, \mathbf{v}_2\}$ is linearly independent. ::: :::: These examples demonstrate how to determine whether a set of vectors is linearly dependent or linearly independent by examining the solutions to the homogeneous vector equation. # Basis and Dimension Basis and dimension are fundamental concepts that provide a way to characterize and quantify vector spaces. A basis provides a minimal set of vectors that can generate the entire vector space, while the dimension quantifies the \"size\" of the vector space. ## Definition of a Basis :::: tcolorbox ::: definition **Definition 7** (Basis). A **basis** for a vector space $\mathcal{V}$ is a set of vectors $\mathcal{B} = \{\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_n\}$ in $\mathcal{V}$ that satisfies two essential conditions: 1. **Linear Independence**: The set $\mathcal{B}$ is linearly independent. This means that no vector in $\mathcal{B}$ can be expressed as a linear combination of the other vectors in $\mathcal{B}$. 2. **Spanning Set**: The set $\mathcal{B}$ spans $\mathcal{V}$. This means that every vector in $\mathcal{V}$ can be expressed as a linear combination of the vectors in $\mathcal{B}$. In other words, $\text{span}(\mathcal{B}) = \mathcal{V}$. ::: :::: A basis can be thought of as a minimal \"scaffolding\" for a vector space. It contains just enough vectors to construct any vector in the space through linear combinations, without any redundancy. ## Finite-Dimensional and Infinite-Dimensional Vector Spaces Vector spaces can be classified based on whether they have a basis with a finite number of vectors. :::: tcolorbox ::: definition **Definition 8** (Finite-Dimensional Vector Space). A vector space $\mathcal{V}$ is called **finite-dimensional** if it possesses a basis consisting of a finite number of vectors. If a vector space does not have a finite basis, it is called **infinite-dimensional**. ::: :::: In this lecture, we primarily focus on finite-dimensional vector spaces, which are commonly encountered in many areas of applied mathematics and computer science. ## Dimension of a Vector Space For finite-dimensional vector spaces, the dimension is a fundamental property that quantifies the number of vectors in any basis for that space. :::: tcolorbox ::: definition **Definition 9** (Dimension). The **dimension** of a finite-dimensional vector space $\mathcal{V}$, denoted by $\text{dim}(\mathcal{V})$, is the number of vectors in any basis for $\mathcal{V}$. A crucial theorem in linear algebra states that *all bases for a finite-dimensional vector space contain the same number of vectors*. Thus, the dimension is a well-defined property of the vector space itself, indicating its \"size\" or degrees of freedom. ::: :::: The dimension provides essential information about the structure of a vector space. For example, in geometric terms, the dimension corresponds to the number of independent directions within the space. ## Examples of Bases and Dimensions Let's examine the standard bases and dimensions for common vector spaces $\mathbb{R}^2$, $\mathbb{R}^3$, and generalize to $\mathbb{R}^n$. ### Basis and Dimension of $\mathbb{R}^2$ :::: tcolorbox ::: example **Example 7** (Basis and Dimension of $\mathbb{R}^2$). The **standard basis** for $\mathbb{R}^2$ is the set $\mathcal{B} = \{\mathbf{e}_1, \mathbf{e}_2\} = \{(1, 0), (0, 1)\}$. 1. **Linear Independence**: $\{\mathbf{e}_1, \mathbf{e}_2\}$ is linearly independent, as shown in [\[example:Linear Independence in \$\\mathbb{R}\^2\$\]](#example:Linear Independence in $\mathbb{R}^2$){reference-type="ref+label" reference="example:Linear Independence in $\\mathbb{R}^2$"}. 2. **Spanning Set**: $\{\mathbf{e}_1, \mathbf{e}_2\}$ spans $\mathbb{R}^2$. Any vector $\mathbf{v} = (x, y) \in \mathbb{R}^2$ can be written as $\mathbf{v} = x\mathbf{e}_1 + y\mathbf{e}_2 = x(1, 0) + y(0, 1) = (x, 0) + (0, y) = (x, y)$. Since $\mathcal{B}$ is linearly independent and spans $\mathbb{R}^2$, it is a basis for $\mathbb{R}^2$. The dimension of $\mathbb{R}^2$ is the number of vectors in this basis, which is 2. Thus, $\text{dim}(\mathbb{R}^2) = 2$. ::: :::: ### Basis and Dimension of $\mathbb{R}^3$ :::: tcolorbox ::: example **Example 8** (Basis and Dimension of $\mathbb{R}^3$). The **standard basis** for $\mathbb{R}^3$ is $\mathcal{B} = \{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\} = \{(1, 0, 0), (0, 1, 0), (0, 0, 1)\}$. 1. **Linear Independence**: It can be shown that $\{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\}$ is linearly independent. 2. **Spanning Set**: $\{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\}$ spans $\mathbb{R}^3$. Any vector $\mathbf{v} = (x, y, z) \in \mathbb{R}^3$ can be written as $\mathbf{v} = x\mathbf{e}_1 + y\mathbf{e}_2 + z\mathbf{e}_3$. Thus, $\mathcal{B}$ is a basis for $\mathbb{R}^3$, and the dimension of $\mathbb{R}^3$ is 3, i.e., $\text{dim}(\mathbb{R}^3) = 3$. ::: :::: ### Basis and Dimension of $\mathbb{R}^n$ :::: tcolorbox ::: example **Example 9** (Basis and Dimension of $\mathbb{R}^n$). Generalizing, the **standard basis** for $\mathbb{R}^n$ is $\mathcal{B} = \{\mathbf{e}_1, \mathbf{e}_2, \dots, \mathbf{e}_n\}$, where $\mathbf{e}_i$ is the vector with a 1 in the $i$-th position and 0 elsewhere. 1. **Linear Independence**: The standard basis for $\mathbb{R}^n$ is linearly independent. 2. **Spanning Set**: The standard basis for $\mathbb{R}^n$ spans $\mathbb{R}^n$. Any vector $\mathbf{v} = (x_1, x_2, \dots, x_n) \in \mathbb{R}^n$ can be written as $\mathbf{v} = x_1\mathbf{e}_1 + x_2\mathbf{e}_2 + \dots + x_n\mathbf{e}_n$. Therefore, $\mathcal{B}$ is a basis for $\mathbb{R}^n$, and the dimension of $\mathbb{R}^n$ is $n$, i.e., $\text{dim}(\mathbb{R}^n) = n$. ::: :::: ## Finding a Basis using Row Reduction Row reduction is a practical method to find a basis for the column space of a matrix, which is the span of its column vectors. This is particularly useful when we are given a set of vectors that span a subspace and we need to find a basis for that subspace. ### Procedure for Finding a Basis using Row Reduction The procedure involves transforming a matrix into row-echelon form and identifying the pivot columns. 1. **Form a Matrix**: Construct a matrix $A$ where the given vectors are the columns of $A$. 2. **Row Reduce to Echelon Form**: Use elementary row operations to transformmatrix $A$ into row-echelon form, say matrix $R$. 3. **Identify Pivot Columns**: Determine the pivot columns in the row-echelon form $R$. Pivot columns are the columns that contain a leading entry (pivot). 4. **Basis Vectors**: The columns in the **original matrix** $A$ that correspond to the pivot columns in the row-echelon form $R$ form a basis for the column space of $A$, $\text{Col}(A) = \text{span}\{\text{columns of } A\}$. This method efficiently extracts a linearly independent subset from the original set of vectors that still spans the same column space, thus providing a basis. An example illustrating this process is shown below. :::: tcolorbox ::: {.example:Row Reduction Basis .example} **Example 10** (Finding a Basis using Row Reduction). Let's find a basis for the subspace spanned by the vectors $\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$, $\mathbf{v}_2 = \begin{pmatrix} 2 \\ 4 \\ 6 \end{pmatrix}$, $\mathbf{v}_3 = \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix}$, and $\mathbf{v}_4 = \begin{pmatrix} 2 \\ 5 \\ 8 \end{pmatrix}$. **Step 1: Form a Matrix** Construct a matrix $A$ with these vectors as columns: $$A = \begin{pmatrix} 1 & 2 & 1 & 2 \\ 2 & 4 & 3 & 5 \\ 3 & 6 & 5 & 8 \end{pmatrix}$$ **Step 2: Row Reduce to Echelon Form** Perform row operations to get the row-echelon form $R$: $$\begin{aligned} \begin{pmatrix} 1 & 2 & 1 & 2 \\ 2 & 4 & 3 & 5 \\ 3 & 6 & 5 & 8 \end{pmatrix} &\xrightarrow{R_2 \leftarrow R_2 - 2R_1, R_3 \leftarrow R_3 - 3R_1} \begin{pmatrix} 1 & 2 & 1 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 2 & 2 \end{pmatrix} \\ &\xrightarrow{R_3 \leftarrow R_3 - 2R_2} \begin{pmatrix} 1 & 2 & 1 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{pmatrix} = R \end{aligned}$$ The row-echelon form is $R = \begin{pmatrix} 1 & 2 & 1 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{pmatrix}$. **Step 3: Identify Pivot Columns** The pivot columns in $R$ are the first and the third columns (containing the leading entries 1 and 1). **Step 4: Basis Vectors** The corresponding columns in the original matrix $A$ are the first and the third columns, $\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$ and $\mathbf{v}_3 = \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix}$. Thus, a basis for the subspace spanned by $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3, \mathbf{v}_4\}$ is $\{\mathbf{v}_1, \mathbf{v}_3\} = \left\{ \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}, \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix} \right\}$. ::: :::: # Rank and the Rank-Nullity Theorem The concept of rank provides a measure of the \"size\" of the image or column space of a matrix, indicating the number of linearly independent columns (or rows). The Rank-Nullity Theorem is a fundamental result that connects the rank of a matrix to the dimension of its null space, offering a deep insight into the structure of linear transformations. ## Rank of a Matrix :::: tcolorbox ::: definition **Definition 10** (Rank of a Matrix). The **rank** of a matrix $A$, denoted as $\text{rank}(A)$, is defined as the dimension of the column space of $A$. Equivalently, it is also the dimension of the row space of $A$. The rank of a matrix represents the maximum number of linearly independent columns (or rows) in the matrix. ::: :::: The rank of a matrix is a non-negative integer that provides crucial information about the matrix's properties and the linear transformation it represents. A higher rank indicates a \"larger\" column space and, consequently, a \"more significant\" linear transformation in terms of dimensionality. ## The Rank-Nullity Theorem :::: tcolorbox ::: theorem **Theorem 3** (Rank-Nullity Theorem). *Let $A$ be an $m \times n$ matrix. Then the sum of the dimension of the null space of $A$ (also called the nullity of $A$) and the rank of $A$ is equal to the number of columns of $A$, which is $n$. In equation form: $$\text{dim}(\text{Nul}(A)) + \text{rank}(A) = n$$ Equivalently, for a linear transformation $\Phi: \mathcal{V}\to \mathcal{W}$ represented by matrix $A$, where $\mathcal{V}$ is $n$-dimensional, the theorem states: $$\text{dim}(\text{ker}(\Phi)) + \text{dim}(\text{Im}(\Phi)) = \text{dim}(\mathcal{V})$$ where $\text{Nul}(A) = \text{ker}(\Phi)$ is the kernel (null space) of $A$ (or $\Phi$), and $\text{rank}(A) = \text{dim}(\text{Col}(A)) = \text{dim}(\text{Im}(\Phi))$ is the rank of $A$ (or the dimension of the image of $\Phi$).* ::: :::: The Rank-Nullity Theorem describes the relationship between the dimensions of the null space and the column space of a matrix, stating that their sum equals the number of columns of the matrix. ### Interpretation and Significance The Rank-Nullity Theorem reveals a fundamental trade-off: for a given matrix or linear transformation, the higher the rank (i.e., the larger the image), the smaller the nullity (i.e., the smaller the kernel), and vice versa, with their dimensions always summing up to the dimension of the domain. - **Dimension of Solution Space**: $\text{dim}(\text{Nul}(A))$ represents the number of free variables in the solution to the homogeneous equation $A\mathbf{x} = \mathbf{0}$. It quantifies the \"size\" of the solution set. - **Dimension of Image**: $\text{rank}(A) = \text{dim}(\text{Col}(A))$ represents the dimension of the space spanned by the columns of $A$, which is the image of the linear transformation. It quantifies the \"reach\" of the transformation. - **Domain Dimension**: $n = \text{dim}(\mathcal{V})$ is the dimension of the input space. The Rank-Nullity Theorem is a powerful tool in linear algebra, providing a way to understand the relationship between the input and output spaces of a linear transformation and the structure of the solutions to linear systems. It is used in various applications, including determining the existence and uniqueness of solutions to linear equations and understanding the properties of linear transformations. # Conclusion In this lecture, we have established the foundational concepts of vector spaces and subspaces, which are crucial for linear algebra. We began by defining vector spaces through their axioms, ensuring a rigorous understanding of their structure. We then explored subspaces, learning how to verify if a subset qualifies as a subspace using the Subspace Test. Key operations of linear combinations and span were introduced, demonstrating how sets of vectors can generate subspaces. We differentiated between linear dependence and independence, concepts essential for understanding the redundancy and efficiency of vector sets. Furthermore, we defined basis and dimension, providing tools to characterize the size and structure of vector spaces, and touched upon the method of row reduction for finding a basis. Finally, we introduced the rank of a matrix and the Rank-Nullity Theorem, which unveils a fundamental relationship between the null space and column space of a matrix. ## Key Takeaways - **Vector Spaces**: Defined by axioms that govern vector addition and scalar multiplication, providing a general framework for linear structures. - **Subspaces**: Subsets of vector spaces that are themselves vector spaces, inheriting operations and properties from the parent space, verifiable by the Subspace Test. - **Linear Combinations and Span**: Linear combinations create new vectors from existing sets, and the span of a vector set forms a subspace, representing all reachable vectors through linear combinations. - **Linear Dependence and Independence**: Linear dependence indicates redundancy within a vector set, while linear independence signifies that each vector is essential for spanning their space. - **Basis and Dimension**: A basis is a minimal, linearly independent spanning set for a vector space, and the dimension is the number of vectors in any basis, quantifying the space's size. - **Rank and Rank-Nullity Theorem**: Rank measures the dimension of the column space, and the Rank-Nullity Theorem relates rank to the dimension of the null space, providing a fundamental insight into linear transformations. ## Further Reading and Next Steps To deepen your understanding and extend your knowledge, consider exploring the following topics: - **Applications to Linear Systems**: Investigate how vector space concepts are used to solve systems of linear equations, including existence and uniqueness of solutions. - **Linear Transformations and Matrices**: Study linear transformations in detail and how they are represented by matrices, including matrix operations and properties. - **Practical Applications**: Explore real-world applications of linear algebra in computer graphics, data analysis, machine learning, and engineering to appreciate the practical significance of these concepts. Understanding these fundamental concepts is crucial for further studies in linear algebra and its applications across various scientific and engineering disciplines.