Find the number of roots of a polynomial using Rouche's Theorem

I don't think that there is a fixed method how you can always proceed. Generally, one tries to find a "dominant" term. In this case, the trick is to consider your polynomial in the circles $|z| < 1 $ and $|z| < 2 $ separately.

For $|z| = 1 $, $$ |z^5 + 1| \le 1 + 1 = 2 < 3 = |3z^2| \, , $$ so you can choose $h(z) = z^5+1$ and $f(z) = 3z^2$ and conclude that $z^5+3z^2+1$ has 2 zeros in $|z| < 1 $.

For $|z| = 2 $, $$ |3z^2 + 1| \le 3 \cdot 4 + 1 = 13 < 32 = |z^5| \, , $$ so you can choose $h(z) = 3z^2 + 1$ and $f(z) = z^5$ and conclude that $z^5+3z^2+1$ has 5 zeros in $|z| < 2 $.