Proof of $H^k(X,\mathbf k) = H^k(X,\mathbb Z) \otimes \mathbf k$

If $S_{\bullet}$ is the singular chain complex for $X$, then $H^i(X,A)$ is computed by the cochain complex $Hom(S_{\bullet},A)$. Now there is natural map $Hom(S_{\bullet},\mathbb Z) \otimes k \to Hom(S_{\bullet},k),$ and hence, passing to cohomology, a map $$H^{\bullet}(X,\mathbb Z)\otimes k \to H^{\bullet}(X,k).$$ This will not be an isomorphism for arbitrary spaces $X$. (E.g. if $X$ is just a disjoint union of a set $I$ of points, then this is the natural map $(\mathbb Z^I)\otimes k \to k^I$, which is not an isomorphism in general if $I$ is infinite.)

However, if the homology groups of $X$ with $\mathbb Z$-coefficients are finitely generated, then this will be an isomorphism, as follows from the universal coefficient theorem for cohomology that you mentioned in your question. (Use the fact that the universal coefficient short exact sequence is natural in $A$; that $Ext(H_{i-1},A)$ is torsion when $A = \mathbb Z$ --- since $H_{i-1}$ is f.g. by assumption --- and vanishes when $A = k$, since $k$ is of char. zero; and that $Hom(H_i,\mathbb Z)\otimes k \cong Hom(H_i,k)$, since $H_i$ is f.g.)

For a compact manifold the homology groups are f.g., and so the result follows.