Why should I care about fields of positive characteristic?
Let's suppose you care about finite groups, one way or another. Then you probably care about the classification of finite simple groups. The bulk of this classification is the groups of Lie type, which were discovered by finding analogues of Lie groups over finite fields.
Finite fields are also (as one might guess) very important in computer science. I'm certainly no expert, but here are some applications I know of:
- Cryptographic protocols like Diffie-Hellman have as their basis the simple fact that it is difficult to invert exponentiation in finite fields.
- The standard way that one factors polynomials over the integers is to apply something like Berlekamp's algorithm to factor them over several finite fields first, then combine the factorizations.
- The classic theorem that IP=PSPACE requires some work over finite fields.
- Elliptic curves over finite fields are used for elliptic curve cryptography.
- I have also been told that vector spaces and varieties over finite fields can be used to construct error-correcting codes. I don't know anything about this, but here is a book on the subject. For the special case of linear codes this leads to a beautiful analogy between lattice sphere packings and error-correcting codes which is described, for example, by Noam Elkies here.
Finally, even if you are only interested in varieties over $\mathbb{C}$ (say), if your variety happens to also be nice and defined over $\mathbb{Z}$ then it can be nice and defined over $\mathbb{F}_p$ for all but finitely many $p$ and you can use the Weil conjectures to compute its Betti numbers by counting. This is particularly easy to do for varieties with nice moduli interpretations like flag varieties.
Edit: You might also be interested in reading Serre's expository article How to use finite fields for problems concerning infinite fields, as well as Manin's Reflections on arithmetical physics. I got the latter link from an excellent answer to an MO question on mirror symmetry over finite fields.
There is a whole paper of Serre on this, which I haven't read; I will explain briefly one of my favorite applications.
One of the really cool applications of this is to the Ax-Grothendieck theorem. Which is a 100% analytic statement: if $P: \mathbb{C}^n \to \mathbb{C}^n$ is an injective polynomial map, then it's surjective. And yet the proof uses finite fields.
How? Well, first note that the theorem is trivial when $\mathbb{C}$ is replaced by a finite field: an injective map on finite sets is surjective. Using a little algebra (and a simple direct limit argument), it follows that the theorem is true for the algebraically closed field of characteristic $p$ which is the algebraic closure of the integers mod $p$.
Now apply to a bit of model theory; the theorem of Robinson says that any statement of first-order logic that is true in an alg. closed field of char. $p$ for each $p$ is true in any algebraically closed field of char 0, in particular the complex numbers. The trick is that the Ax-Grothendieck theorem can be phrased as a collection of (infinitely many) statements in first-order logic. And each is true in the algebraic closure of the finite fields. Whence, by Robinson's theorem, it's true in $\mathbb{C}$!
This would be cheating if Robinson's theorem were something obscure. In fact, however, it is a direct application of the compactness theorem for first-order logic.
For another proof via the Nullstellensatz (but based on finite fields, still), see Terry Tao's post.
If you are interested in algebraic geometry over C, here is another reason. A basic technique in birational geometry is the bend and break, which roughly amounts to taking any curve on a projective variety and deforming it until it becomes a union of curves of lower genus. Eventually this technique can be used to produce rational curves (under appropriate hypothesis).
This is a major tool, and has been used for instance to study the geometry of Fano varieties, in particular to prove that they are uniruled, or to prove Hartshorne conjecture on the characterization of projective space by the positivity of its tangent bundle.
The fact is, to make the trick work you have to have a curve whose space of embedded deformations is big enough. This is easy to achieve in positive characteristic: the curve itself may not deform, but if you compose the inclusion with a sufficiently high power of the Frobenius morphism, the map that you obtain will have enough deformations. To prove the result in characteristic 0, a technique of reduction to characteristic p is used.
You can't avoid finite fields if you do complex algebraic geometry, just as you can't avoid (and it is an advantage to utilize) variables that square to zero. Some major techniques work by reduction mod $p$ and there aren't always substitutes that accomplish the same directly in characteristic 0.
A lot of algorithms work by reducing modulo many primes and then patching together the results (Chinese remainder theorem). For the analysis of what at happens the primes you need to understand finite fields, and in some cases, p-adic fields.
Error-correcting codes have a fundamental significance as constructions of "uniformly distributed" or "well spaced" objects in many dimensions. This is true irrespective of their additional commercial, cryptographic or computational significance. And it happens that the comprehensible part of error-correcting code theory is the part related to linear algebra, number theory, and algebraic geometry over finite fields. To the extent that error-correcting codes are seen to be a model for phenomena in nature or its mathematical idealizations (crystals, packings, coverings, spin glasses, combinatorial phase transitions, etc) then routine use of finite fields is unavoidable, even if you don't care about number theory per se.
The construction of the real field from the rationals using Cauchy sequences can be mimicked to construct other (characteristic zero) complete fields not isometric to the reals. Namely, instead of starting out with the metric defined by the usual (archimedean) absolute value, one can consider the $p$-adic metric ($p$ a fixed prime) and proceed in the same lines. A theorem of Ostrowski says that up to equivalence these are in fact the only ways to complete $\Bbb Q$.
Being complete, the fields ${\Bbb Q}_p$ thus obtained can be used to develop an analytic theory which is similar to the classical theory "over $\Bbb R$" but has some subtle differences.
A source of difference is that the fields ${\Bbb Q}_p$ have--unlikely $\Bbb R$--a natural subring, the radius $1$ sphere centered in $0$, a.k.a. the $p$-adic integers $\Bbb Z_p$, which is a local ring with residue field isomorphic to the field with $p$ elements.
This is a clue to the fact that "$p$-adic analysis" is deeply intertwined with the theory of finite fields.