Reasons to prefer Little Snitch over the built-in firewall

Little Snitch offers three features that aren't available in MacOS' built-in ipfw firewall. (It does this by loading a custom kernel module.)

1. Little Snitch allows you to block outgoing connections; the MacOS firewall only blocks incoming connections. Handy if you're running some untrusted program and aren't sure what it's going to do, or if you want to disable a program for updating itself, or if you want to prevent access to a specific resource. Also, I suspect many people use Little Snitch to block pirated software from checking their license.
2. Little Snitch lets you configure the firewall per application, not just address or port. Ie: you can configure it so one web browser can access a web site but not another.
3. Little Snitch also monitors network traffic on a per-application basis. It's easy on MacOS to see how much bandwidth you're using but much harder to see which program is using that bandwidth. The Little Snitch shows network usage for each application, albeit in a limited way.

That being said, I don't think Little Snitch is "must have" software; these features are fairly esoteric. There are also several alternatives: TCPBlock and glowworm for the firewall and Rubbernet (now defunct) for the monitoring.

2016 Update: MacOS now has the per-application monitoring built into Activity Monitor.

Basic differences

The basic task of the MacOSX Firewall is to monitor incoming network connections. HandsOff and LittleSnitch also allow to monitor outgoing network connections. The latter functionality is essential for various reasons like spyware and privacy.

Because LittleSnitch does not monitor incoming connections (unlike HandsOff!) it cannot replace the MacOX Firewall but is a companion to optimize network security.

Important features

Unlike the MacOSX Firewall both programs offer a much higher degree of differentiation when defining rules to network traffic:

• Rules can be applied for a limited time (e.g. until applications quits, until reboot, forever)
• Rules can block user defined (sub-)domains and ports for applications and processes

You basically define your own firewall step-by-step using such rules.

Both programs also include a network monitor that can show detailed information about your network traffic on the desktop.

Important to know

Keep in mind that these programs do not offer 100% network security. Little Snitch cannot monitor software that uses it's own kernel-extension. Moreover, there is no implementation for a behavioral analysis of potentially malicious software. (source, German)

Left picture: Real Time Monitor. Right picture: Rules set in the preferences.