High School
SAT
ACT
University
MCAT
LSAT
DAT
Wize University Linear Algebra Textbook > Orthogonality

Orthonormal Basis and Gram-Schmidt Process

Column Space and Null Space (Range and Kernel)Previous Section
Orthogonal Complement and Orthogonal Projection (COMING SOON)Next Section
Popular Courses
Find My Course
0:00 / 0:00

Orthonormal Basis

Orthogonal Set

A set of vectors is said to be an orthogonal set if every pair of vectors in the set is orthogonal.
Wize Concept
Recall that two vectors u⃗\vec u and v⃗\vec v are orthogonal if and only if their dot product is 0.
u⃗⋅v⃗=0  ⟺  u⃗⊥v⃗\vec u \cdot \vec v =0 \iff \vec u \perp\vec v

Example
The set {[1−11],[−1−10],[−112]}\left\{ \begin{bmatrix} 1\\ -1\\ 1\\ \end{bmatrix} , \begin{bmatrix} -1\\ -1\\ 0\\ \end{bmatrix} , \begin{bmatrix} -1\\ 1\\ 2\\ \end{bmatrix} \right\} is an orthogonal set.
PAGE BREAK

Orthonormal Set

A set of vectors is said to be orthonormal if:
  1. It is an orthogonal set
  2. Every vector in the set is a unit vector (a vector of length 1)
Example
{[1/3−1/31/3],[−1/2−1/20],[−1/61/62/6]}\left\{ \left[\begin{array}{r} 1/\sqrt3\\ -1/\sqrt3\\ 1/\sqrt3\\ \end{array}\right] , \left[\begin{array}{r} -1/\sqrt2\\ -1/\sqrt2\\ 0\\ \end{array}\right] , \left[\begin{array}{r} -1/\sqrt6\\ 1/\sqrt6\\ 2/\sqrt6\\ \end{array}\right] \right\} is an orthonormal set.
Here we have turned each of the vectors from the previous example into a normal vector.

Create unit vectors by normalizing each vector v⃗i\vec v_i in the set (divide by its length): 1∥v⃗i∥v⃗i\dfrac{1}{\|\vec v_i\|}\vec v_i.
Wize Concept
The standard basis in every real vector space Rn\mathbb{R}^n is an orthonormal basis.
E.g. {[10],[01]}\left\{ \begin{bmatrix} 1\\ 0\\ \end{bmatrix}, \begin{bmatrix} 0\\ 1\\ \end{bmatrix} \right\} is an orthornormal basis for R2\reals^2.

0:00 / 0:00

Example: Orthonormal Sets

Part A)

Show that {(1,0,1,0),(0,1,0,1),(−1,0,1,0),(0,−1,0,1)}\{(1,0,1,0),(0,1,0,1),(-1,0,1,0),(0,-1,0,1)\} is an orthogonal set.
Let v⃗1=(1,0,1,0), v⃗2=(0,1,0,1), v⃗3=(−1,0,1,0), v⃗4=(0,−1,0,1)\vec v_1=(1,0,1,0),\ \vec v_2=(0,1,0,1),\ \vec v_3=(-1,0,1,0),\ \vec v_4=(0,-1,0,1), then:
v⃗1⋅v⃗2=0v⃗1⋅v3=0v⃗1⋅v⃗4=0\vec v_1\cdot \vec v_2=0 \qquad \vec v_1\cdot v_3=0\qquad \vec v_1\cdot \vec v_4=0
v⃗2⋅v⃗3=0v⃗2⋅v⃗4=0\vec v_2\cdot \vec v_3=0 \qquad \vec v_2\cdot \vec v_4=0
v⃗3⋅v⃗4=0\vec v_3\cdot \vec v_4=0
Since each pair or vectors is orthogonal, the given set is orthogonal.

Part B)

Normalize each vector to obtain an orthonormal set.
Each vector in the set has a norm of 2\sqrt{2}, so multiply every vector by the scalar 12\dfrac{1}{\sqrt{2}}.
The orthonormal set is:
{(12,0,12,0),(0,12,0,12),(−12,0,12,0),(0,−12,0,12)}\left\{ \left(\dfrac{1}{\sqrt{2}},0,\dfrac{1}{\sqrt{2}},0\right), \left(0,\dfrac{1}{\sqrt{2}},0,\dfrac{1}{\sqrt{2}}\right), \left(-\dfrac{1}{\sqrt{2}},0,\dfrac{1}{\sqrt{2}},0\right), \left(0,-\dfrac{1}{\sqrt{2}},0,\dfrac{1}{\sqrt{2}}\right) \right\}
0:00 / 0:00

Gram-Schmidt Process

The Gram-Schmidt process transforms any basis into an orthonormal basis using projections.

Wize Concept
Recall: Given two vectors u⃗\vec u and v⃗\vec v, the projection of u⃗\vec u onto v⃗\vec v is a vector parallel to v⃗\vec v:
projv⃗u⃗=(u⃗⋅v⃗v⃗⋅v⃗)v⃗\boxed{\quad \text{proj}_{\normalsize \vec v}\vec u= \left(\dfrac{\vec u\cdot \vec v}{\vec v \cdot\vec v}\right)\vec v \quad}
Subtracting this vector from u⃗\vec u, we can create a vector that is orthogonal to v⃗\vec v:
perpv⃗u⃗=u⃗−projv⃗u⃗\boxed{\quad \text{perp}_{\normalsize \vec v} \vec u = \vec u - \text{proj}_{\vec v}\vec u \quad}

PAGE BREAK

Gram-Schmidt Process

Given a basis {u⃗1,u⃗2,u⃗3,…,u⃗n}\{\vec u_1,\vec u_2,\vec u_3,\dots,\vec u_n\}, we can create an orthonormal basis {v⃗1,v⃗2,v⃗3,…,v⃗n}\{\vec v_1,\vec v_2,\vec v_3,\dots,\vec v_n\} as follows:
Step 1
Let v⃗1=u⃗1\vec v_1=\vec u_1.
Step 2
  • Project u⃗2\vec u_2 onto v⃗1\vec v_1: projv⃗1u⃗2\qquad \colorTwo{\text{proj}_{\normalsize \vec v_1}\vec u_2}
  • Subtract the projection from u⃗2\vec u_2 to find the perpendicular component: v⃗2=u⃗2−projv⃗1u⃗2\qquad \vec v_2 = \vec u_2 - \colorTwo{\text{proj}_{\normalsize \vec v_1}\vec u_2}
Step 3
  • Project u⃗3\vec u_3 onto the the previously found vectors v⃗i\vec v_i and add them all up: projv⃗1u⃗3+projv⃗2u⃗3\qquad \colorTwo{\text{proj}_{\normalsize \vec v_1}\vec u_3 + \text{proj}_{\normalsize \vec v_2}\vec u_3}
  • Subtract this sum of projections from u⃗3\vec u_3 to find the perpendicular component: v⃗3=u⃗3−(projv⃗1u⃗3+projv⃗2u⃗3)\qquad \vec v_3=\vec u_3 - \colorTwo{(\text{proj}_{\normalsize \vec v_1}\vec u_3 + \text{proj}_{\normalsize \vec v_2}\vec u_3)}
  • Repeat Step 3 until you've created the vector v⃗n\vec v_n: v⃗n=u⃗n−(projv⃗1u⃗n+projv⃗2u⃗n+projv⃗3u⃗n+…+projv⃗n−1u⃗n)\qquad \vec v_n=\vec u_n - (\text{proj}_{\normalsize \vec v_1}\vec u_n + \text{proj}_{\normalsize \vec v_2}\vec u_n + \text{proj}_{\normalsize \vec v_3}\vec u_n+\ldots+ \text{proj}_{\normalsize \vec v_{n-1}}\vec u_n)
Step 4
Normalize: for every vector v⃗i\vec v_i, rewrite as 1∥vi∥v⃗i\dfrac{1}{\|v_i\|} \vec v_ i
0:00 / 0:00

Example: Gram-Schmidt Process

Let U=span{(2,1,−1),(−5,1,3),(1,1,9)}U=\text{span}\{(2,1,-1),(-5,1,3),(1,1,9)\} be a subspace of R3\mathbb{R}^3.
Use the Gram-Schmidt process to find an orthonormal basis for UU.
Define u⃗1=(2,1,−1), u⃗2=(−5,1,3), u⃗3=(1,1,9)\vec u_1=(2,1,-1),\ \vec u_2=(-5,1,3),\ \vec u_3=(1,1,9)
We now apply the Gram-Schmidt process:
Step 1
Let v⃗1=u⃗1=(2,1,−1)\vec v_1=\vec u_1=(2,1,-1)
Step 2
Project u⃗2\vec u_2 onto v⃗1\vec v_1, and subtract this projection from u⃗2\vec u_2:
projv⃗1u⃗2=u⃗2⋅v⃗1v⃗1⋅v⃗1v⃗1=(−5,1,3)⋅(2,1,−1)(2,1,−1)⋅(2,1,−1)(2,1,−1)=−126(2,1,−1)=(−4,−2,2)\text{proj}_{\normalsize \vec{v}_1}\vec{u}_2 =\dfrac{\vec{u}_2\cdot\vec{v}_1}{\vec{v}_1\cdot\vec{v}_1}\vec{v}_1 =\dfrac{(-5,1,3)\cdot(2,1,-1)}{(2,1,-1)\cdot(2,1,-1)}(2,1,-1) =\dfrac{-12}{6}(2,1,-1)=(-4,-2,2)
v⃗2=u⃗2−projv⃗1u⃗2=(−5,1,3)−(−4,−2,2)=(−1,3,1)\vec v_2=\vec u_2-\text{proj}_{\normalsize \vec v_1}\vec u_2=(-5,1,3)-(-4,-2,2)=(-1,3,1)
We can check that this new vector v⃗2\vec v_2 is indeed orthogonal to v⃗1\vec v_1:
v⃗1⋅v⃗2=(2,1,−1)⋅(−1,3,1)=0\vec v_1\cdot \vec v_2=(2,1,-1)\cdot(-1,3,1)=0
Step 3
Project u⃗3\vec u_3 onto both v⃗1\vec v_1 and v⃗2\vec v_2, and subtract these projections from u⃗3\vec u_3:
projv⃗1u⃗3=u⃗3⋅v⃗1v⃗1⋅v⃗1v⃗1=(1,1,9)⋅(2,1,−1)(2,1,−1)⋅(2,1,−1)(2,1,−1)=−66(2,1,−1)=(−2,−1,1)\text{proj}_{\normalsize \vec v_1}\vec u_3=\dfrac{\vec u_3\cdot\vec v_1}{\vec v_1\cdot\vec v_1}\vec v_1 =\dfrac{(1,1,9)\cdot(2,1,-1)}{(2,1,-1)\cdot(2,1,-1)}(2,1,-1) =\dfrac{-6}{6}(2,1,-1) =\left(-2,-1,1\right)
projv⃗2u⃗3=u⃗3⋅v⃗2v⃗2⋅v⃗2v⃗2=(1,1,9)⋅(−1,3,1)(−1,3,1)⋅(−1,3,1)(−1,3,1)=1111(−1,3,1)=(−1,3,1)\text{proj}_{\normalsize \vec v_2}\vec u_3 =\dfrac{\vec u_3\cdot\vec v_2}{\vec v_2\cdot\vec v_2}\vec v_2 =\dfrac{(1,1,9)\cdot(-1,3,1)}{(-1,3,1)\cdot(-1,3,1)}(-1,3,1) =\dfrac{11}{11}(-1,3,1) =(-1,3,1)
v⃗3=u⃗3−(projv⃗1u⃗3+projv⃗2u⃗3)=(1,1,9)−(−2,−1,1)−(−1,3,1)=(4,−1,7)\vec v_3 =\vec u_3-(\text{proj}_{\normalsize \vec v_1}\vec u_3+\text{proj}_{\normalsize \vec v_2}\vec u_3) =(1,1,9)-(-2,-1,1)-(-1,3,1) =(4,-1,7)
We now have an orthogonal basis for UU:
{(2,1,−1),(−1,3,1),(4,−1,7)}\{(2,1,-1),(-1,3,1),(4,-1,7)\}
Step 4
To normalize, we need to know the length/norm of each vector:
∥(2,1,−1)∥=6,∥(−1,3,1)∥=11,∥(4,−1,7)∥=66\lVert (2,1,-1) \rVert = \sqrt{6},\qquad \lVert (-1,3,1) \rVert = \sqrt{11},\qquad \lVert (4,-1,7)\rVert = \sqrt{66}
Divide each vector in the basis by its norm to obtain an orthonormal basis for UU:
{(26,16,−16),(−111,311,111),(466,−166,766)}\left\{ \left( \dfrac{2}{\sqrt{6}},\dfrac{1}{\sqrt{6}},-\dfrac{1}{\sqrt{6}} \right), \left( -\dfrac{1}{\sqrt{11}},\dfrac{3}{\sqrt{11}},\dfrac{1}{\sqrt{11}} \right), \left( \dfrac{4}{\sqrt{66}},-\dfrac{1}{\sqrt{66}},\dfrac{7}{\sqrt{66}} \right) \right\}
checklist
Mark Yourself Question
  1. Grab a piece of paper and try this problem yourself.
  2. When you're done, check the "I have answered this question" box below.
  3. View the solution and report whether you got it right or wrong.

Practice: Gram-Schmidt Process

Use the Gram-Schmidt process to create an orthonormal basis for the following subspace WW of R4\mathbb{R}^4.
W=span{u⃗1,u⃗2,u⃗3}W=\text{span}\{\vec u_1,\vec u_2,\vec u_3\} where u⃗1,u⃗2,u⃗3\vec u_1,\vec u_2, \vec u_3 are the columns of the matrix:
A=[136112134110]A=\left[\begin{array}{rrr} 1&3&6\\ 1&1&2\\ 1&3&4\\ 1&1&0 \end{array}\right]

Note: WW is the column space of the matrix AA.
Extra Practice
Span
Let B:={(1,1,0,2),(0,1,0,−1),(0,0,1,1)}B:=\{(1,1,0,2),(0,1,0,-1),(0,0,1,1)\} , and U:=spanBU:=spanB .
a) Find a pairwise orthogonal set of nonzero vectors OO so that U=spanOU=spanO .
b) Write the Fourier expansion of (0,0,1,0)(0,0,1,0) in terms of the basis OO .
19.4F_Final_Builder_Ch_17.2_Orthogonal_Complement_$\tkco{eg1}$_$\key{Final}$_Builder_$\tkcth{17.2.}\tkcf{1}$_$\key{Quiz}$
Show that the set of vectors
{[121][−11−1][−303]} \left\{ \colvecth{1}{2}{1} \colvecth{-1}{1}{-1} \colvecth{-3}{0}{3} \right\}
is a basis for R3\mathbb{R}^3using *only* the fact that the vectors are all perpendicular to one another (i.e. no points would be awarded for using Gaussian elimination).
Consider the subspace W={(x,y,z)∈R3  ∣  x−y+3z=0}W=\Big\{(x,y,z)\in \mathbb{R}^3 \; | \; x-y+3z=0 \Big\} of R\mathbb{R}.
Find a basis for WW that contains the vector v⃗=[0  3  1]\vec{v}=[0 \; 3 \; 1].
Find orthogonal complement of WW in R3\mathbb{R}^3.
Gram-Schmidt process
Consider the basis B={(1,3),(2,1)}B=\{(1,3),(2,1)\} of R2\mathbb{R}^2
Use the Gram-Schmidt process to change BB into an orthonormal basis B′B'
Gram-Schmidt process
Use the Gram-Schmidt proces to find an orthonormal basis for U=Span{(1,2,1,2),(3,3,3,4),(7,6,5,3)}U=\text{Span}\{(1,2,1,2),(3,3,3,4),(7,6,5,3)\}
Practice: Gram-Schmidt Process
Use the Gram-Schmidt process to create an orthonormal basis for the subspace WW of R4\mathbb{R}^4:
W=Span({u⃗1,u⃗2,u⃗3})W=\text{Span}\Big(\{\vec u_1,\vec u_2,\vec u_3\}\Big) where u⃗1,u⃗2,u⃗3\vec u_1,\vec u_2, \vec u_3 are the columns of the matrix:
A=[136112134110]A=\left[\begin{array}{rrr} 1&3&6\\ 1&1&2\\ 1&3&4\\ 1&1&0 \end{array}\right]
Practice Question: Gram-Schmidt
Find an orthonormal basis of null(A)null(A) , where A=[−12312−45−1]A=\left[\begin{array}{rrrr}-1&2&3&1\\2&-4&5&-1\end{array}\right] .
Orthonormal Basis and Gram-Schmidt Process
Example: Gram-Schmidt Process
Let U=span{(2,1,−1),(−5,1,3),(1,1,9)}U=\text{span}\{(2,1,-1),(-5,1,3),(1,1,9)\} be a subspace of R3\mathbb{R}^3.
Use the Gram-Schmidt process to find an orthonormal basis for UU.
Orthonormal Basis
Example: Orthonormal Sets
Part A)
Show that {(1,0,1,0),(0,1,0,1),(−1,0,1,0),(0,−1,0,1)}\{(1,0,1,0),(0,1,0,1),(-1,0,1,0),(0,-1,0,1)\} is an orthogonal set.
Basis of Orthogonal Complement
Find an orthonormal basis for the orthogonal complement of the intersection of the planes 2x+y−z=02x+y-z=0 and 4x−3y=04x-3y=0 in R3\mathbb{R}^3 .
If we apply the Gram-Schmidt process to the vectors (3,0,4,0)(3, 0, 4, 0), (−1,0,7,0)(−1, 0, 7, 0), and (0,1,3,0)(0, 1, 3, 0) in that order, to obtain the orthonormal vectors u1,u2,u3u_1, u_2, u_3 in that order, then what isu3u_3?
Projection onto a Subspace
Calculate projspan{(1,1,0,1),(1,0,1,1)}(4,−2,−1,−1)proj_{span\{(1,1,0,1),(1,0,1,1)\}}(4,-2,-1,-1) .
B={[11−1],[101],[1−2−1]}B=\left\{\begin{bmatrix}1\\1\\-1\end{bmatrix},\begin{bmatrix}1\\0\\1\end{bmatrix},\begin{bmatrix}1\\-2\\-1\end{bmatrix}\right\} is an orthogonal basis of R3R^3 . Which of the following is the coordinate vector of [311]\begin{bmatrix}3\\1\\1\end{bmatrix}relative to the basis?
Find an orthogonal basis for R3R^3containing the vector [12−2]\begin{bmatrix}1\\2\\-2\end{bmatrix}