A first-principles approach to rotation in $\mathbb{R}^3$ given an orthogonal matrix with $\det A = 1$.
Of great importance in linear algebra and geometry is the concept of rotation about an axis. In this writeup, I will focus solely on the behavior of rotation in $\mathbb{R}^3$, though the concepts can be extended to $\mathbb{R}^n$. We will come to see that the parity of $n$ is important in characterizing the behavior of rotation. In any case, we are going to begin with a simple setting in $\mathbb{R}^3$ with limited assumptions on a linear transformation $T_A$ with matrix $A \in M_3(\mathbb{R})$.
As an aside, the “interesting applications” of such matrices are not the true motivaton for this writeup. My professor offered this challenge problem (Prop. 1 below) as a way to get out of doing a homework assignment, so here goes.
Proposition 1. If $A \in M_3(\mathbb{R})$ is orthogonal with $\det A = 1$, then $A$ is a rotation matrix.
Proof.
As hinted above, we are going to work this out using first principles. We begin with the fact that since $A$ is orthogonal, $A^{-1}$ exists and $A^{-1} = A^T$. We are also going to state an auxillary theorem from [1] that will be prevalent in our proof.
Theorem 1.1 For $A \in M_3(\mathbb{C})$, the following are equivalent:
$A$ is orthogonal;
The rows of $A$ form an orthonormal basis for $\mathbb{C}^n$;
The columns of $A$ form an orthonormal basis for $\mathbb{C}^n$;
For $u, v \in \mathbb{C}^n$, we have $\langle Au, Av \rangle = \langle u, v \rangle$;
For $u \in \mathbb{C}^n$, we have $\|Au\| = \|u\|$.
We now begin with the proof with an important claim.
Claim 1.1. $1 \in \operatorname{spec}A$.
Proof.
By Theorem 1.1, we have that $A$ orthogonal implies that, for any $u \in \mathbb{R}^3$, $u$ satisfies $\|Au\|= \|u\|$. By dimension of $A$ and the fundamental theorem of algebra, we can have maximally 3 distinct eigenvalues in $\mathbb{C}$, and since complex eigenvalues come in conjugate pairs, $\lambda \in \operatorname{spec} A$ with $\lambda \in \mathbb{C} \implies \overline{\lambda} \in \operatorname{spec} A$. Thus we must have either 1 or 3 strictly real eigenvalues. More, $\lambda \in \operatorname{spec} A$ implies that $|\lambda| = 1$ by Theorem 1.1. We now suppose that $\lambda = -1$ and break into 2 disjoint, exhaustive cases:
$\lambda_2, \lambda_3 \in \operatorname{spec} A$ with $\lambda_2 \in \mathbb{C}$ and $\lambda_3 = \overline{\lambda_2}$. Then
$$\det A = +1 = \lambda_1 \lambda_2 \lambda_3 = -|\lambda_2|^2,$$
where $|\lambda_2|$ is strictly positive (and thus it's square is too) since $A$ invertible implies that none of our eigenvalues are zero. Thus $-|\lambda_2|^2 < 0$ contradiction, and it must be that $\lambda_1 = +1$.
$\lambda_2, \lambda_3 \in \operatorname{spec} A$ with $\lambda_2, \lambda_3 \in \mathbb{R}$. As stated above, we know that $|\lambda_2| = |\lambda_3| = 1$. Then $\det A = 1+ = \lambda_1 \lambda_2 \lambda_3 \implies \lambda_2 \lambda_3 = -1$. Thus WLOG if $\lambda_2 = -1$, we must have $\lambda_3 = 1$.
We have shown that, no matter the composition of the spectrum of $A$, at least one eigenvalue is equal to 1, and thus $1 \in \operatorname{spec}A$.
$\blacksquare$
Ok, cool. From here on, we assume WLOG that only one such $\lambda \in \operatorname{spec} A$ is equal to $+1$ since otherwise $\lambda_1 = \lambda_2 = \lambda_3$ and we have that $A = \operatorname{Id}$ which is trivially a rotation by zero degrees. Now to the meat of the proof.
Let $E$ be the eigenspace associated with $\lambda = 1 \in \operatorname{spec} A$ such that $E = \{u \in \mathbb{R}^3 \mid Au = u\}$ which we've assumed above has $\dim E = 1$. (As an aside, we know that $\dim E > 0$ since, by the very existence of an eigenvalue, there is at least one nonzero $u \in \mathbb{R}^3$ satisfying $Au = u$). Then since $\mathbb{R}^3$ is a normed, inner product space and $E < \R^3$ is a subspace, we have that the orthogonal complement $E^\perp$ exists and $\operatorname{codim} E^\perp = 1 \implies E^\perp$ is a subspace (a plane embedded) in $\mathbb{R}^3$. For any $u \in \mathbb{R}^3$, consider it's orthogonal decomposition $u = \operatorname{proj}_{E^\perp}(u) + \operatorname{proj}_E(u)$ and define $w = \operatorname{spec}_{E^\perp}(u)$. By linearity of $A$ and the construction of $E$, we have
$$Au = Aw + A\operatorname{proj}_E(u) = Aw + \operatorname{proj}_E(u).$$
Let's focus just on $w$ and $Aw$ then since the projection of $u$ in $E$ is invariant under $A$.
Claim 1.2. $E^\perp$ is also invariant under $A$.
Proof.
Consider some $u \in E$ and $v \in E^\perp$. By definition of orthogonally complimentary spaces, $\langle u, v \rangle = 0$. But from Theorem 1.1, we know that $A$ orthogonal implies that $\langle u, v \rangle = \langle Au, Av \rangle = 0$ and thus since we know that $E$ is invariant under $A$, $Au \in E \implies Av \in E^\perp$. So $v \in E^\perp \implies Av \in E^\perp$.
$\blacksquare$
Claim 1.3. We can re-express $A$ as a block matrix under change of basis as
$$\operatorname{Rep}_C(A) = \begin{bmatrix}1 & \textbf{0} \\ \textbf{0} & \textbf{R}\end{bmatrix}$$
such that $\textbf{R} \in M_2(\mathbb{R})$ and $\textbf{R}$ is orthogonal and automorphic over $\operatorname{Rep}_C(E^\perp) \cong \mathbb{R}^2$.
Proof.
Consider a unit vector $u \in E$. Further consider that since $E^\perp$ is a subspace with $\dim E^\perp = 2$, we know there exists a (not necessarily unique) set of vectors $\{v, w\} \subset E^\perp$ such that $v$ and $w$ form an orthonormal basis for $E^\perp$. By definition of orthogonally complimentary spaces, we have then that ${u, v, w}$ are all pair-wise orthogonal and of unit length, and thus form a basis for $\R^3$. Let $B = \operatorname{Id} = [\textbf{e}_1\ \textbf{e}_2\ \textbf{e}_3]$ be the standard basis for $\mathbb{R}^3$ and $C = [u\ v\ w]$ be our newly constructed basis. Since the columns of both $B, C$ are orthonormal, both matrices are orthogonal and thus invertible by Theorem 1.1 again. Then since we have that
$$ AC = \begin{bmatrix}
Au & Av & Aw
\end{bmatrix} = \begin{bmatrix}
u & Av & Aw
\end{bmatrix},$$
we know that in representation $C$ (that is, $C^{-1}AC = C^TAC$), $A$ takes on the form
$$ \operatorname{Rep}_C(A) = C^TAC = \begin{bmatrix}
1 & \textbf{0} \\
\textbf{0} & \textbf{R}
\end{bmatrix}$$
Some quick intuition behind the presence of the zeroes. First, since $A, C$, and $C^T$ are all orthogonal, so is their composition, meaning rows/columns of $\operatorname{Rep}_C(A)$ are orthonormal. Second, we know that both $E$ and $E^\perp$ are invariant under $A$ and thus $\operatorname{Rep}_A(C)$ as well. Thus for any $x \in E^\perp$, we know that $\langle Tx, u \rangle = 0$. Trivially, by structure of the block matrix in representation $C$, we know that $\textbf{R}$ is endomorphic over $\operatorname{Rep}_C(E^\perp)$. The isomorphic property of $\textbf{R}$ follows from the orthonormality of it's rows/columns, and thus it's invertibility in $M_2(\mathbb{R})$.
$\blacksquare$
Let $T = \operatorname{Rep}_C(A)$ as in Claim 1.3. We've shown thus far that $u \in E \implies Tu = u$ and that $\textbf{R}$ within $T$ is an orthogonal matrix whose determinant is also one since $\det A = \det C \det T \det C^T = \det T = 1 \cdot \det \textbf{R} = 1$. We provide a quick auxillary claim.
Claim 1.4. $A$ orthogonal implies that $A$ is diagonalizable.
Proof.
Stuff probably should go here, not sure though...
$\blacksquare$
We now arrive at the main result of the proof.
Claim 1.5. $\textbf{R}$ is the rotation matrix in $\mathbb{R}^2$.
Proof.
We have that $\textbf{R}$ is orthogonal and has determinant 1. As shown earlier on, $\lambda \in \operatorname{spec}\textbf{R} \implies |\lambda| = 1$ and thus $\lambda$ lies somwhere on the complex unit circle such that there exists a $\theta \in [0, 2\pi)$ where $\lambda = e^{i\theta}$. Moreover, $\lambda \in \operatorname{spec}\textbf{R} \implies \overline{\lambda} \in \operatorname{spec} \textbf{R}$. Suppose $\theta = 0$ such that $\lambda_1 = \lambda_2 = 1$. By diagonalizability of orthogonal matrices, we know that there must exists two distinct eigenvectors, and in fact $\textbf{R} = I$ rotation by zero degrees. Similarly, if $\theta = \pi$ where $\lambda_1 = \lambda_2 = -1$, we ge that $\textbf{R} = -I$, the rotation by $180$ degrees. Now we can assume WLOG that $\lambda_1 = \overline{\lambda_2}$ are distinct complex eigenvalues with parameter $\theta$ and distinct eigenvectors $v$ and $\overline{v}$, respectively. Thus $\textbf{R}v = e^{i\theta} v$ and $\textbf{R}\overline{v} = e^{-i\theta}\overline{v}$. Suppose that $v = v_1 + i v_2$ with $v_1, v_2 \in \R^2$. Then we have that
$$
\textbf{R}v = (\cos\theta + i\sin\theta)v \\
\textbf{R}\overline{v} = (\cos\theta - i\sin\theta)\overline{v}
$$
by Euler's identity and the fact that $\sin$ is odd. Re-expressing this as
$$
\textbf{R}v_1 + i\textbf{R}v_2 - i\textbf{R}v_2 + \textbf{R}v_1 = 2v_1\cos\theta - 2v_2\sin\theta \\
\textbf{R}v_1 + i\textbf{R}v_2 - \textbf{R}v_1 + i\textbf{R}v_2 = 2iv_1\sin\theta + 2iv_2\cos\theta
$$
and thus $\textbf{R}v_1 = v_1\cos\theta - v_2\sin\theta$ and $\textbf{R}v_2 = v_1\sin\theta + v_2\cos\theta$. Further, we know that $\langle v_1, v_2 \rangle = 0$ since otherwise, $v = (1+ic)v_1$ and thus $\textbf{R}v_1 = e^{i\theta}v_1$ which is a contradiction since $e^{i\theta}$ would leave the image in $\mathbb{C}^2$, not $\mathbb{R}^2$. Thus $\{v_1, v_2\}$ forms an orthogonal basis for $\mathbb{R}^2$. Now consider any $u \in \mathbb{R}^2$. We can express $u = av_1 + bv_2$ and thus,
$$
\textbf{R}u = a\textbf{R}v_1 + b\textbf{R}v_2 = (a\cos\theta + b\sin\theta)v_1 + (b\cos\theta -a\sin\theta)v_2.
$$
If $T$ is the change of basis such that $v_1 \mapsto \textbf{e}_1$ and $v_2 \mapsto \textbf{e}_2$, then we get that
$$
T\textbf{R}\begin{bmatrix}
a \\ b
\end{bmatrix} = T\begin{bmatrix}
a\cos\theta - b\sin\theta \\
a\sin\theta + b\cos\theta
\end{bmatrix}
$$
and thus $\textbf{R}$ is the rotation matrix in $\mathbb{R}^2$.
$\blacksquare$
Wrapping things up now, we've shown that, for the same $C = [u\ v\ w]$ as before,
$$
\operatorname{Rep}_C(A) = \begin{bmatrix}
1 & \textbf{0} \\
\textbf{0} & \textbf{R}
\end{bmatrix}.
$$
Consider now any $x \in \R^3$. If we apply orthogonal decomposition such that
$$
x = \operatorname{proj}_E(x) + \operatorname{proj}_{E^\perp}(x) = c_1u + c_2v + c_3w,
$$
we have that
$$
\operatorname{Rep}_C(A)x = \operatorname{Rep}_C(A) \begin{bmatrix}
c_1 \\ c_2 \\ c_3
\end{bmatrix} = \begin{bmatrix}
c_1 \\ \textbf{R}\begin{bmatrix}
c_2 \\ c_3
\end{bmatrix}
\end{bmatrix} = \begin{bmatrix}
c_2 \\
c_2\cos\theta - c_3\sin\theta \\
c_2\sin\theta + c_3\cos\theta
\end{bmatrix}
$$
and thus
$$
x = \operatorname{Rep}_C(A)^{-1} \circ \begin{bmatrix}
c_2 \\
c_2\cos\theta - c_3\sin\theta \\
c_2\sin\theta + c_3\cos\theta
\end{bmatrix}
$$
which corresponds to a rotation in the $yz$-plane in representation $C$ of $\mathbb{R}^3$ or, in our standard basis, a rotation of vector $x$ around the vector $u$ by $\theta$ radians in the counterclockwise direction.
$\blacksquare$
As a result of laziness, I’ve only typed up the first, rough draft of my proof here. It’s guarunteed to contain errors. My more complete, final version can be found here! Anyway, thanks for reading my (probably only) blog post of 2025.
