Prove the Curvature Tensor is a Tensor

A $(k,l)$-tensor field $A$ on smooth manifold $M$ is a smooth section $$ A : M \longrightarrow T^{(k,l)}TM, $$ where $T^{(k,l)}TM = \coprod_{p \in M} \underbrace{T_pM \otimes \cdots \otimes T_pM}_{k \text{ times}} \otimes \underbrace{T^*_pM \otimes \cdots \otimes T^*_pM}_{l \text{ times}} $.

Any tensor field has property that for any smooth vector fields $X_1,\dots,X_l$ and smooth covector fields $\omega^1,\dots,\omega^k$ we have a smooth function $$ A(\omega^1,\dots,\omega^k,X_1,\dots,X_l) : M \longrightarrow \mathbb{R} $$ defined as $A(\omega^1,\dots,\omega^k,X_1,\dots,X_l)(p) = A_p(\omega^1|_p,\dots,\omega^k|_p,X_1|_p,\dots,X_l|_p) \in \mathbb{R}$. So $(k,l)$-tensor field $A$ induces a map $$ \tilde{A} : \underbrace{\mathfrak{X}^*(M)\times\cdots\times\mathfrak{X}^*(M)}_{k \text{ times}} \times \underbrace{\mathfrak{X}(M) \times \cdots \times \mathfrak{X}(M)}_{l \text{ times}} \longrightarrow C^{\infty}(M). $$ An important fact is that this map is multilinear over $C^{\infty}(M)$. It turns out that this in fact characterize tensor fields.

$\textbf{Tensor Characterization Lemma}$ ( Lee's Introduction to Smooth Manifolds ) A map \begin{equation} \tau : \underbrace{\mathfrak{X}^*(M)\times\cdots\times\mathfrak{X}^*(M)}_{k \text{ times}} \times \underbrace{\mathfrak{X}(M) \times \cdots \times \mathfrak{X}(M)}_{l \text{ times}} \longrightarrow C^{\infty}(M) \qquad \color{blue}{(\star)} \end{equation} is induced by a smooth $(k,l)$ tensor field as above iff this map is multilinear over $C^{\infty}(M)$.

In the proof of the above lemma, the tensor induce from the multilinear map $\tau : \mathfrak{X}^*(M) \times \cdots \mathfrak{X}^*(M) \times \mathfrak{X}(M)\times \cdots \mathfrak{X}(M) \longrightarrow C^{\infty}(M)$ is a tensor field $A : M \to T^{(k,l)}TM$ defined as $$ A_p(w^1,\dots,w^k,v_1,\dots,v_l):= \tau(\omega^1,\dots,\omega^k,X_1,\dots,X_l)(p) $$ where $\omega^i\in\mathfrak{X}^*(M)$ is any smooth extension of $w^i\in T^*_pM$ and $X_i \in \mathfrak{X}(M)$ is any smooth extension of $v_i \in T_pM$, for each $i$ (the map $A$ independent of the extensions).

Because of this lemma, we often identify $A$ with its induced map $\tilde{A}$.

In your case above the map is \begin{equation} R : \mathfrak{X}(M)^* \times \mathfrak{X}(M) \times \mathfrak{X}(M) \times \mathfrak{X}(M) \rightarrow C^{\infty}(M) \end{equation} defined as $$R(\omega,X,Y,Z) := \omega(\nabla_X\nabla_Y Z - \nabla_Y\nabla_X Z - \nabla_{[X,Y]}Z)$$ is defines a $(3,1)$-tensor field if you can show that $R$ is linear over $C^{\infty}(M)$ on each arguments. This including the first and the fourth argument which is not mentioned before in the comments above. The first argument is obvious $$ R(f\omega,\cdot,\cdot,\cdot) = (f\omega)(\cdot) = f\,\omega(\cdot) $$ The second and the third and the fourth is similar, it follows from \begin{align*} \nabla_{fX_1+gX_2}&= f\nabla_{X_1} + g \nabla_{X_2} \quad \textbf{(Linearity)}\\ \nabla_X(fY) &= f\nabla_X Y + (Xf)Y \quad \textbf{(Leibniz Rule)}\\ [fX,Y] &= f[X,Y] -(Yf)X \end{align*} For example, \begin{align*} R(\omega,fX,Y,Z) &= \omega (\nabla_{fX} \nabla_Y Z - \nabla_Y \nabla_{fX} Z-\nabla_{[fX,Y]}Z) \\ &= \omega (f\nabla_X \nabla_Y Z - \nabla_Y (f\nabla_XZ ) - \nabla_{f[X,Y]-(Yf)X}Z) \\ &=\omega (f\nabla_X \nabla_Y Z - f\nabla_Y\nabla_XZ - \require{cancel}{\cancel{(Yf)\nabla_XZ}} - f\nabla_{[X,Y]} Z + {\cancel{(Yf)\nabla_XZ}}) \\ &= f R(\omega,X,Y,Z). \end{align*}

This lemma is oftenly used in Riemannian geometry text implicitly. You can see the proof in Lee's book here p.318.

Beside the type of maps in $\color{blue}{(\star)}$ above, another but similar way to spot a disguised tensor field is through these kind of maps $$ \tau_0 : \underbrace{\mathfrak{X}^*(M)\times\cdots\times\mathfrak{X}^*(M)}_{k \text{ times}} \times \underbrace{\mathfrak{X}(M) \times \cdots \times \mathfrak{X}(M)}_{l \text{ times}} \longrightarrow \color{red}{\mathfrak{X}(M)}. $$ If this map is multilinear over $C^{\infty}(M)$ then we can define a map $$ \tau : \underbrace{\mathfrak{X}^*(M)\times\cdots\times\mathfrak{X}^*(M)}_{\color{red}{k+1 \text{ times}}} \times \underbrace{\mathfrak{X}(M) \times \cdots \times \mathfrak{X}(M)}_{l \text{ times}} \longrightarrow C^{\infty}(M) $$ as $\tau(\omega^{1},\dots,\omega^{k}, \omega^{k+1},X_1,\dots,X_l) = \tau_0(\omega^{1},\dots,\omega^{k},X_1,\dots,X_l)(\omega^{k+1})$, which is also multilinear over $C^{\infty}(M)$. Therefore $\tau_0$ induced by a smooth $(k+1,l)$-tensor field (as its induced map $\tau$ induce them in the Tensor Characterization Lemma above) iff $\tau_0$ is multilinear over $C^{\infty}(M)$. Other respective variants of $\tau_0$ can be deduced the same way.