How to make mobs change textures based on their custom name

This is not possible with vanilla, though OptiFine has it as a feature. Here's a documentation of it.

The custom texture for the mob must be named vindicator<num>.png inside the assets\minecraft\optifine\random\entity\illager folder, where <num> is a number (starts at 1.)

Inside the same folder, you'll also need a .properties file with the same name as the vanilla texture (meaning it would be vindicator.properties.) Here's how its contents would look like:

name.<num>=Enchanter

• <num> being the same number as the custom texture has in its name.

This should now display the custom texture to the vindicator with the name "Enchanter"