Can I connect my HDMI amplifier as audio only?

After some more digging it doesn't seem possible to completely remove the additional display.

However a trick I was not aware of (see Use multiple displays with your Mac) is to both extend the desktop (I use both my MBP LCD and an external display), and mirror the amplifier "display" to one of the real displays. When arranging displays hold the Option key and drag the unwanted display onto a real display. I mirror it with my MBP LCD. Although it alters the screen resolution of the LCD upon mirroring, you can just change it back to what it was before.

This way the cursor cannot find its way to the amp display, nor can any windows inadvertently open on it.