Properties

We want to use the formula for similar matrices: $B=P^{-1}AP$.

First, find $P^{-1}$:

$P^{-1} = P^T =\dfrac{1}{\sqrt{2}}\begin{bmatrix}1&1\\-1&1\end{bmatrix}$

Now we can find a similar matrix using the formula:

$\begin{aligned} P^{-1}AP &= \dfrac{1}{\sqrt{2}}\begin{bmatrix}1&1\\-1&1\end{bmatrix} \begin{bmatrix}3&2\\4&1\end{bmatrix} \dfrac{1}{\sqrt{2}}\begin{bmatrix}1&-1\\1&1\end{bmatrix}\\[1.5em] &=\dfrac{1}{2}\begin{bmatrix}7&3\\1&-1\end{bmatrix}\begin{bmatrix}1&-1\\1&1\end{bmatrix}\\[1.5em] &=\frac{1}{2}\begin{bmatrix}10&-4\\0&-2\end{bmatrix}\\[1.5em] &=\begin{bmatrix}5&-2\\0&-1\end{bmatrix}=B \end{aligned}$

Since $B$ is upper triangular, its eigenvalues are on the main diagonal.

Therefore, the eigenvalues of $B$ are $\lambda_1 = 5$ and $\lambda_2 = -1$. Since $A \sim B$, these are also the eigenvalues of $A$.

Since $L$ is linear, it can be represented by a matrix $A$ such that $L(\vec v) = A\vec v$.

Let's start by finding the matrix $A$ that induces this transformation:

$L\left(\begin{array}{c}x\\y\\z\end{array}\right)=\left(\begin{array}{c}-2x-y-3z\\x+z\\x+y+2z\end{array}\right) =\left(\begin{array}{r}-2x-1y-3z\\ 1x+0y+1z\\1x+1y+2z\end{array}\right) =\left[\begin{array}{rrr}-2&-1&-3\\1&0&1\\1&1&2\end{array}\right]\left[\begin{array}{c}x\\y\\z\end{array}\right]$

$\implies A=\left[\begin{array}{rrr}-2&-1&-3\\1&0&1\\1&1&2\end{array}\right]$

Since $\vec x$ is an eigenvector of $L$, it must also be an eigenvector of $A$. Let's find its eigenvalue:

$L(\vec x) = A\vec x = \left[\begin{array}{rrr}-2&-1&-3\\1&0&1\\1&1&2\end{array}\right] \left[\begin{array}{r}1\\-1\\0\end{array}\right] = \left[\begin{array}{r}-1\\1\\0\end{array}\right] = \textcolor{green}{\mathbf{-1}}\vec x$

So $\vec x$ is an eigenvector with associated eigenvalue $\lambda=\textcolor{green}{\mathbf{-1}}$.

Then notice that the vector $(L \circ L \circ L \circ L \circ L)(\vec x)$ is equivalent to $A^5\vec x$, since applying $L$ five times is equivalent to multiplying by $A$ five times.

$L(L(L(L(L(\vec x))))) = A^5 \vec x$

and by a property of eigenvalues: $A^5 \vec x = \lambda^5\vec x$

Therefore:

$(L \circ L \circ L \circ L \circ L)(\vec x) =A^5\vec x = \lambda^5\vec x = (\textcolor{green}{\bm{-1}})^5\vec x = -\vec x = -\begin{bmatrix} 1\\ -1\\ 0\\ \end{bmatrix} = \boxed{\ \begin{bmatrix} -1\\ 1\\ 0\\ \end{bmatrix}\ }$

$L$ is invertible if $A$ is invertible, and $A$ is invertible if and only if 0 is not one of its eigenvalues.

The question tells us $A=\left[\begin{array}{rrr}-2&-1&-3\\1&0&1\\1&1&2\end{array}\right]$ has eigenvalues $\lambda_1=1$ and $\lambda_2=-1$ (since $A$ and $L$ must have the same eigenvalues).

From the properties of eigenvalues, the sum of the eigenvalues must equal the trace of matrix $A$. In other words:

$\begin{aligned} \lambda_1+\lambda_2+\lambda_3 &=a_{11}+a_{22}+a_{33}\\[0.5em] 1+(-1)+\lambda_3 &=(-2)+0+2\\[0.5em] \lambda_3&=0 \end{aligned}$

The third eigenvalue of $A$ is 0, so $A$ is not invertible, therefore $L$ is not invertible.