How to check whether an ideal is a prime (or maximal) ideal?

$\newcommand{\Z}{\mathbb{Z}}$

This works for principal ideals pretty well, and it was how I was shown to compute the class group. Let me see if I remember this right...

One way is to translate the problem. You are looking to see if the ideal $I$ is prime in an imaginary quadratic ring. For $d \equiv 1,2$ modulo $4$, this is $\frac{\Z[x]}{x^2+d}$; for $d \equiv 3$ modulo $4$, it's $\frac{\Z[x]}{x^2 + x + \frac{d+1}{4}}$.

The ideal $I$ is prime if $R/I$ is an integral domain (no zero divisors). The trick then is to interchange the quotients of the rings. So for example, in $d=5$, say you're checking to see if $I=(2)$ is prime. Then $R=\frac{\Z[x]}{x^2+5}$. Now by interchanging the quotients this is equivalent to seeing whether $(x^2+5)=(x^2+1)$ is prime in $\frac{\Z[x]}{2}$. In this case it is not, since $(x^2+1) = (x+1)^2$. If you translate this back, since $x$ "represents" $\sqrt{-5}$, you get that the ideal $(2) = (1+\sqrt{-5})^2$.

To check if an ideal is prime, one way is to look at the quotient $R / I$; the ideal $I$ is prime if and only if $R / I$ is an integral domain.

To expand a bit on that: say you have two elements $(x, y)$, and you want to know if the ideal $I$ they generate is prime. This ideal $I$ certainly contains the elements $xx^*$ and $y y^*$ (where * is the Galois "conjugation" automorphism of R). These are in $\mathbb{Z}$, so they have a highest common factor; let's say that is $k$.

Then $kR$ is an ideal of $R$ contained in $I$. Moreover, the ring $R / k R$ is finite, and you can now describe $R / I$ by a finite computation, because it's a quotient of $R / kR$. In particular, you can check whether or not $R / I$ is a field.