High School
SAT
ACT
University
MCAT
LSAT
DAT

19.4F_Final_Builder_Ch_17.3_Least_Squares_$\tkco{eg4}$_$\key{Final}$_Builder_$\…

Related Topics

Wize University Linear Algebra Textbook > Orthogonality

Orthogonal Complement and Orthogonal Projection (COMING SOON)

3 Activities

checklist
Mark Yourself Question
  1. Grab a piece of paper and try this problem yourself.
  2. When you're done, check the "I have answered this question" box below.
  3. View the solution and report whether you got it right or wrong.
The figure below illustrates the relationship between the Im (A‾)\bct{\text{Im}\, (\A)} and b⃗\bco{\color{cyan}\vb} for the system given by:
{x−2y=−42x−4y=7⇔[  1−2    2−4  ][abc] ⁣ ⁣ ⁣ ⁣⏟:= A‾[xy]=[−47][abc]⏟:= b⃗ \left\{ \begin{array}{rr} x - 2y \quad =& -4 \\ 2x - 4y \quad =& 7 \end{array} \right. % \qquad \Leftrightarrow \qquad % \underbrace{ \begin{bmatrix} \; 1 & -2 \; \\ \; 2 & -4 \; \end{bmatrix} \vphantom{\colvecth{a}{b}{c}\!\!\!\!} }_{:= \, \bct{\A}} % \colvec{x}{y} % = % \underbrace{ \colvec{-4}{7} \vphantom{\colvecth{a}{b}{c}} }_{:=\, \bco{\color{cyan}\vb}}



Let b⃗Im(A‾)=proj⇀Im(A‾)(b⃗)\vb_{_{\textrm{Im}(\A)}} = \overrightharpoon{\text{proj}}_{_{\textrm{Im}(\A)}}(\vb), i.e. the (vector) component of b⃗\vb which is in the image of A‾\A. Identify this vector on the figure; we'll call this component b⃗∣∣\vb_{_{||}} for short.

N.B. the vector isn't included in the figure, so by "identify" you'd have to draw it. I'd strongly suggest you make a quick sketch of your own and add the vector to that sketch: this is an essential concept to be comfortable with, so it's worth actually doing.

Let b⃗⊥Im(A‾)=b⃗−b⃗∣∣\vb_{\perp_{_{\textrm{Im}(\A)}}} = \vb - \vb_{_{||}}, i.e. b⃗⊥\vb_{\perp} is the (vector) component of b⃗\vb which is not in the image of A‾\A. Identify this vector on the figure; we'll call this component b⃗⊥\vb_{\perp} for short.
More Orthogonal Complement and Orthogonal Projection (COMING SOON) Questions:
The projection of a vector b⃗\vb onto the image of a set of vectors {v⃗1,v⃗2}\big\{ \vv_1, \vv_2 \big\} is equal to the sum of the projection of the vector b⃗\vb onto each of the individual vectors, i.e.:
proj⇀Im(v⃗1,v⃗2)(b⃗)=proȷ⇀v1⃗ ⁣(b⃗)+proȷ⇀v2⃗ ⁣(b⃗) \overrightharpoon{\textrm{proj}}_{_{\textrm{Im}(\vv_1,\vv_2)}}(\vb) = \proj{b}{v_1} + \proj{b}{v_2}
Span
Let B:={(1,1,0,2),(0,1,0,−1),(0,0,1,1)}B:=\{(1,1,0,2),(0,1,0,-1),(0,0,1,1)\} , and U:=spanBU:=spanB .
a) Find a pairwise orthogonal set of nonzero vectors OO so that U=spanOU=spanO .
b) Write the Fourier expansion of (0,0,1,0)(0,0,1,0) in terms of the basis OO .
19.4F_Final_Builder_Ch_17.2_Orthogonal_Complement_$\tkco{eg4}$_$\key{Final}$_Builder_$\tkcth{17.2.}\tkcf{4}$_
Suppose U={v⃗1, … , v⃗n}U = \left\{ \vv_1,\, \dots\,,\, \vv_n \right\} is an orthogonal basis for Rn\mathbb{R}^{n}, then any vector w⃗\vw in Rn\mathbb{R}^{n} can be written as a linear combination of these vectors, i.e.
w⃗=∑inci v⃗i=c1 v⃗1+c2 v⃗2+ … +cn v⃗n. \vw = \sum_{i}^{n} c_i \, \vv_i = c_1 \, \vv_1 + c_2 \, \vv_2 + \, \dots \, + c_n \, \vv_n.
Find an expression for the constants c1c_1, c2c_2, …\dots, cnc_n in terms of w⃗\vw and the vectors in UU.
19.4F_WML_6_$\tkco{eg10}$_$\key{Final}$_Builder_$\tkcth{17.1.}\tkct{11}$_
Find the set of all vectors perpendicular to the vectors {a⃗1, a⃗2 }={[1101], [0111]}\left\{ \va_1,\, \va_2\ \right\} = \left\{ \colvecf{1}{1}{0}{1}, \, \colvecf{0}{1}{1}{1} \right\}
Consider the subspace W={(x,y,z)∈R3  ∣  x−y+3z=0}W=\Big\{(x,y,z)\in \mathbb{R}^3 \; | \; x-y+3z=0 \Big\} of R\mathbb{R}.
Find a basis for WW that contains the vector v⃗=[0  3  1]\vec{v}=[0 \; 3 \; 1].
Find orthogonal complement of WW in R3\mathbb{R}^3.
19.4F_WML_6_$\tkco{eg7}$_$\key{Final}$_Builder_$\tkcth{17.1.}\tkct{7}$_
Find a basis for the set of all vectors that are perpendicular to both {[1102], [1010]}\left\{ \colvecf{1}{1}{0}{2}, \, \colvecf{1}{0}{1}{0} \right\}
19.4F_WML_6_$\tkco{eg13}$_$\key{Final}$_Builder_$\tkcth{17.1.}\tkct{14}$_
Show that any vector v⃗\vv in the span of the set
V={v⃗1, v⃗2, v⃗3}={[102], [214], [112]} V = \left\{ \vv_1,\, \vv_2,\, \vv_3 \right\} = \left\{ \colvecth{1}{0}{2}, \, \colvecth{2}{1}{4}, \, \colvecth{1}{1}{2} \right\}
will be perpendicular to any vector w⃗\vw in the span of the orthogonal complement W=V⊥W = V^{\perp} of the set.
19.4F_WML_6_$\tkco{eg12}$_$\key{Final}$_Builder_$\tkcth{17.1.}\tkct{13}$_
Describe the orthogonal complement of the matrix whose columns are given by the vectors {v⃗1, v⃗2, v⃗3}={[102], [214], [112]}\left\{ \vv_1,\, \vv_2,\, \vv_3 \right\} = \left\{ \colvecth{1}{0}{2}, \, \colvecth{2}{1}{4}, \, \colvecth{1}{1}{2} \right\} by finding a basis for the orthogonal complement, and describing the span of the basis in geometric terms.
Example: Fourier Expansion
Example: Fourier Expansion
Write the vector (1,1,1)(1,1,1) as a linear combination of {(1,0,1),(1,1,−1),(−1,2,1)}\{(1,0,1),(1,1,-1), (-1,2,1)\}
Basis of Orthogonal Complement
Find an orthonormal basis for the orthogonal complement of the intersection of the planes 2x+y−z=02x+y-z=0 and 4x−3y=04x-3y=0 in R3\mathbb{R}^3 .
Orthogonal Complements
Suppose UU is a subspace of Rn\mathbb{R}^n . Prove that (U⊥)⊥=U(U^\perp)^\perp=U .
Orthogonal Complement
If UU is a subspace of Rn\mathbb{R}^n , show that dim(U)+dim(U⊥)=ndim(U)+dim(U^\perp)=n .
Basis of Orthogonal Complement
Find a basis for the orthogonal complement of the intersection of the planes 2x+y−z=02x+y-z=0 and 4x−3y=04x-3y=0 in R3\mathbb{R}^3 .
Projection onto a subspace
Suppose u⃗,v⃗∈Rn\vec u,\vec v\in\mathbb{R}^n satisfy u⃗⋅v⃗=0,∣∣u⃗∣∣=∣∣v⃗∣∣=1\vec u\cdot\vec v=0, ||\vec u||=||\vec v||=1, and define U:=span{u⃗,v⃗}U := span\{\vec u,\vec v\} .
Using the definition that projUx⃗=A(ATA)−1ATx⃗proj_U\vec x=A(A^TA)^{-1}A^T\vec x , where A=[u⃗v⃗]A=\left[\begin{array}{rr}\vec u&\vec v\end{array}\right] , show that for all x⃗∈Rn\vec x\in\mathbb{R}^n we have that projUx⃗=proju⃗x⃗+projv⃗x⃗proj_U\vec x=proj_{\vec u}\vec x+proj_{\vec v}\vec x .
Projection onto a Subspace
Suppose S:={a⃗1,...,a⃗k}S:=\{\vec a_1,...,\vec a_k\} is a set of orthonormal vectors in Rn\mathbb{R}^n , U:=span(S)U:=span(S) , and suppose that the n×kn\times k matrix AA is defined by A:=[a⃗1a⃗2...a⃗k]A:=\left[\begin{array}{rrrr}\vec a_1&\vec a_2&...&\vec a_k\end{array}\right] .
Show that AATAA^T is the matrix of the linear transformation: Rn→Rnx⃗→projUx⃗\begin{array}{l}\mathbb{R}^n\rightarrow\mathbb{R}^n\\ \vec x\rightarrow proj_U\vec x\end{array} .