Well Fitted/Custom Fit armor perks: If the armor type of the shield is different, will it affect these perks?
Solution 1:
I just tested this out on my character and can safely say that no, the shield being Heavy Armor will not remove the bonuses received from Custom Fit, Wind Walker, or Matching Set. Those bonuses are applied as long as you have Light Armor equipped to your Head, Chest, Hands, and Feet. So, boots, gloves, helm, and armor. The shield has nothing to do with those perks, so you're good!
Solution 2:
No, shields do not count towards armor sets. However, if, for example, you were a light armor user with all the light armor perks, heavy armor shields would not receive any bonuses from Agile Defender.