How does one show that two functors are *not* isomorphic?

The functors $FV=(V^*)^{\otimes 2}$ and $G(V)=(V^{\otimes2})^*$ on the category of finite fimensional vector spaces are isomorphic. The map $\phi:(V^*)^{\otimes2}\to(V^{\otimes2})^*$ such that $\phi(f\otimes g)(v\otimes w)=f(v)g(w)$ is a natural transformation (defined on the whole category of vector spaces) which in the category of f.d. vector spaces is an isomorphism of functors.

Similarly, $\Lambda(V^*)$ and $(\Lambda V)^*$ are canonically isomorphic on the category of f.d. vector spaces.

Here is one way to construct families of endofunctors $F, G$ of $\text{FinVect}$ that can't be distinguished on the basis of dimension counts. First, consider the family of endofunctors $V \mapsto S^n(V)$, and second, consider the family of endofunctors $V \mapsto \Lambda^n(V)$. The first sends vector spaces of dimension $d$ to vector spaces of dimension ${d+n-1 \choose n}$ while the second sends vector spaces of dimension $d$ to vector spaces of dimension ${d \choose n}$. Both of these form bases for the space of integer-valued polynomials in one variable ($d$), hence together there are many linear dependences between them. The smallest one is that

$${d+1 \choose 2} = {d \choose 2} + d$$

which implies that $F(V) = S^2(V)$ and $G(V) = \Lambda^2(V) \oplus V$ can't be distinguished by their dimension counts.

Nevertheless, they are not isomorphic. The reason is that for any endofunctor $F$, the vector space $F(V)$ naturally admits the structure of a $\text{GL}(V)$-representation, and $S^2(V)$ and $\Lambda^2(V) \oplus V$ are not isomorphic as $\text{GL}(V)$-representations. More simply, you can consider the action of a diagonal matrix on $S^2(V)$ and on $\Lambda^2(V) \oplus V$ and verify that the traces (which are symmetric functions) are different.

In fact I think every endofunctor (edit: defining a polynomial function between Hom spaces, see Steve's comment) of $\text{FinVect}$ is equivalent to a direct sum of Schur functors, so they can all be distinguished in this way up to isomorphism. (I need some mild hypothesis on the base field; I think characteristic zero suffices.)

I would like to address the question posed in the title in its full generality, rather than discussing single cases.

Two parallel functors are simply objects in a functor category. Citing Awodey:

One way to prove that two objects are not isomorphic is to use “invariants”: attributes that are preserved by isomorphisms. If two objects differ by an invariant they cannot be isomorphic. Generalized elements provide an easy way to define invariants....

So if you are convinced that 2 functors are not isomorphic, try to use that feeling to provide invariants or generalized elements in the functor category that back up your intuition.

If the functor category is locally small, the above procedure amounts to using the Yoneda embedding: 2 objects in a locally small category are isomorphic if and only if their Yoneda embeddings are isomorphic.

This is certainly a slick and systematic procedure, but it may not be that easy.