Detect all entities with tag "xyz", if none exist, perform an action

You have to use a scoreboard or tag for that. Let's say you have a dummy armour stand somewhere that holds your game scores and other stuff, called "Horst". You have to first give the armour stand a tag if there is any entity of the type you want to detect:

/execute @e[type=<mob_you_want>] ~ ~ ~ scoreboard players tag @e[name=Horst] add entitiesLeft

It can also be anything else, you just need an entity that can have tags and that can be tested for tags. You could just as well use all players (@a).

Afterwards, you can execute based on that tag:

/execute @e[name=Horst,tag=!entitiesLeft] ~ ~ ~ <command>

Comparators are not needed. In general redstone isn't really needed for command block contraptions anymore. You can just make all commands that should depend on this be executed from this /execute command.

In 1.13 you can do all of this in one command:

/execute unless entity @e[type=<mob_you_want>] run <command>

Again, repeat for every command that should depend on there being no entites left.