How do I efficiently kill Handymen?

The most effective vigor to use against Handymen is oddly Murder of Crows. While this vigor does not directly damage the Handymen, it causes them to stop and swat in the air for a couple of seconds, leaving their heart critical open. Playing through on 1999, this was literally the only tactic that allowed me to kill them. In the same way you can perma-stun Zealot of the Lady (the crow enemy) with Shock Jockey, you can almost permalock the Handymen with Murder of Crows.

Also, if you are playing through on 1999, be aware that there is a new chapter before almost every single Handyman, so if you find yourself dying too many times and want to use the lost money for upgrades, it may be better to back out to the menu while dying as the Handymen will recover all of their health when you respawn.

Devils kiss and the shotgun do a lot of damage. Upgrade the shotgun and try to shoot him in the heart for extra damage.

One thing you can do is use a undertow vigor and a shotgun or hand cannon.Get up close to him and hit him with the vigor and he will be stunned,so then shoot him with the hand cannon and shotgun while he is stunned.