How can I rebind Caps lock to delete/backspace?

You may want to try Seil (formerly PCKeyboardHack). I use it to bind Caps Lock to Right-Ctrl (not plain Ctrl), so that I can bind Caps Lock to Meta in Emacs (while Ctrl remains Ctrl, and other keys are kept to the usual behavior).

If you're on macOS Sierra or later, and don't mind using the command line, you can do it with a technique I saw on Hang Lu Su's blog:

$ hidutil property --set '{"UserKeyMapping":[{"HIDKeyboardModifierMappingSrc":0x700000039,"HIDKeyboardModifierMappingDst":0x70000002A}]}'

In her linked post she gets into other things as well, like making sure this binding survives restarts or mapping any key. Here are the Apple docs explaining the technique used as well as the codes to use for any key.

I realize the question and its original answers are older than macOS Sierra, but perhaps this can be useful for people stumbling across it more recently.

Seil is now obsolete and has been replaced by Karabiner-Elements. https://karabiner-elements.pqrs.org