Why do people use bouncycastle instead of Java's built in JCE provider? What is the difference?

BouncyCastle has many more cipher suites and algorithms than the default JCE provided by Sun.

In addition to that, BouncyCastle has lots of utilities for reading arcane formats like PEM and ASN.1 that no sane person would want to rewrite themselves.

Bouncy Castle is Australian in origin, and therefore is not subject to the Export of cryptography from the United States.

It is useful if you are outside the United States and you need to manage key sizes grater than permitted by such that restriction. In that case you are not permitted to use software from United States for that.

On server or desktop, I don't see any reason to use BC unless you have to deal with some legacy ciphers or formats not supported by Sun JCE.

However, many JREs don't come with a JCE provider, like on mobile or embedded environments. BC comes handy in such cases.