Is this matrix always orthogonal?

I was reading about orthogonal matricies and noticed that the $2 \times 2$ matrix $$\begin{pmatrix} \cos(\theta) & \sin(\theta) \\ -\sin(\theta) & \cos(\theta) \end{pmatrix} $$ is orthogonal for every value of $\theta$ and that every $2\times 2$ orthogonal matrix can be expressed in this form. I then wondered if this can be generalized to any smooth paramtrization of the unit circle. More precisely, let $x(t)$ and $y(t)$ be a smooth parametrization of the unit circle for all $t$ in some interval $I \subseteq \Bbb R$ such that $|\langle x(t),y(t) \rangle| = 1$ for all $t \in I$. Is the matrix $$A= \begin{pmatrix}x(t)&y(t)\\x'(t) & y'(t) \end{pmatrix} $$ necessarily orthogonal? At first I thought yes, but I'm having trouble proving it. Letting $v = \langle x(t), y(t) \rangle$, it suffices to show three things: 1) $|v| = 1$, 2) $v \cdot v' = 0$, and 3) $|v'| = 1$. (1) follows straight from how $x(t)$ and $y(t)$ were defined. (2) can be obtained by differentiating the equation $x(t)^2 + y(t)^2 = 1$: \begin{align*} &\frac{d}{dt} \Big[ x(t)^2 + y(t)^2 \Big]= 0 \\ &\implies 2x(t)x'(t) + 2y(t)y'(t)= 0 \\ &\implies v \cdot v' = 0. \end{align*} But I couldn't find a way to prove (3). Now I am unsure whether (3) is true at all. Is $A$ even orthogonal in the first place? Any hints would be much appreciated.

First, not every orthogonal matrix is of the stated form, only rotation matrices. There are orthogonal reflection matrices, with determinant $-1$: $$ \left[ \begin{array}{@{}rr@{}} \cos\theta & \sin\theta \\ \sin\theta & -\cos\theta \end{array}\right]. $$

Second, your conjecture is not true: Only unit speed parametrizations have the stated property, because the rows of an orthogonal matrix form an orthonormal set (in the obvious sense), and particularly must be unit vectors.

The third point implies that the parametrization goes over the circle at a constant speed $1$, which is, of course, false in general.

Take for example $v(t)=\langle \sin t^2,\cos t^2\rangle $ for $t\in[0,\sqrt{2\pi})$.

In general, (3) is not true. Since $\left\vert v \right\vert^2 = x(t)^2 + y(t)^2 = 1$ , we can write $$x(t) = \cos \theta(t), \quad y(t) = \sin \theta(t)$$ for some function $\theta(t)$. But computing gives $\left\vert\dot v\right\vert^2 = \dot\theta(t)^2$, so the condition $|\dot v|^2 = 1$ (which just says that $v$ is a unit speed parameterization) forces $\dot\theta(t) = \pm 1$, and this condition yields just the matrices of the given form (the rotations). Note that a composition of any rotation with a reflection is another reflection and hence orthogonal, but these reflections cannot be of the form in the question: Reflections have determinant $-1$ but matrices of the given form have determinant $(\cos \theta)(\cos \theta) - (\sin \theta)(-\sin \theta) = +1$.

No, that is not the case. Ad you show, the rows in your $A$ will necessarily be orthogonal to each other, and the first row has norm $1$ -- but the norm of the second row is not necessarily $1$.

If you scale each of $x'(t)$ and $y'(t)$ in the second row by $\dfrac{1}{\sqrt{x'(t)^2+y'(t)^2}}$, you do get all of $SO(2)$, assuming that your $x'(t)$ and $y'(t)$ never vanish at the same time.