prevent launchctl from starting a anonymous process

That isn't actually launchctl creating a new process; it's most likely dnsmasq itself. If it fork()s into the background, launchd can't tell; then when it talks to launchd, launchd creates an anonymous entry for it. Make sure dnsmasq doesn't background itself.

(The - listed for the regular launchctl list output indicates that the original dnsmasq process exited.)