What happens if you get negative defense?

Solution 1:

The results are in! I equipped an Oreo on a 0-defense character for a total of -4 defense. The damage I took while I had it on was increased by 4 and was displayed in the regular red color. Purple text means armor-piercing damage, not just 0 defense; otherwise it would be purple when your shots damaged pirates or other easy enemies.

So basically, it does what you'd think.