Is there any reason to care that my wife is cheating on me?
Solution 1:
For single player games there is only one reason to do so.
Bastards who are identified as such are born with the Bastard trait, making them unable to inherit from either parent, and preventing them from passing on either parent's dynasty. (see Bastard).
So, it is a way to control your inheritance. Having many sons in an agnatic-gavelkind realm is bad, but finding out that one is a bastard, you can easily kick him from sucession.
For multiplayer games, there is a second possibility to utilize this. If your bastard-sons real father/mother is your liege, the liege has the possibility to legatimize the child for little cost (20 piety). This would change the childs dynasty from the vassal ones (yours) to the dynasty of your liege, effectively bearing a great danger to your sucession.
Solution 2:
Yes, genetics.
If your ruler has some great traits like the genius trait, there is a chance it will be passed on to your children at a 15% chance.
If your wife is sleeping with a random character the chance that she is sleeping with someone else with positive traits is significantly smaller, and the chance the offspring is born with the genius trait is much lower (0.5%). Even worse, her lover could have negative traits which the child could inherit.
In addition, if your wife is sleeping with one of her family members the chance of getting a negative trait like inbred or lunatic becomes bigger.
Of course if your ruler is an ugly imbecile with a stutter having your wife cheat on you could be a good thing as there's a chance she'll collect some good DNA to insert into your line of succession.
Check here for more details on how breeding works.
Solution 3:
Some diseases, especially the Great Pox, are more likely to be spread between lovers. So if her lover has it, she will likely get it soon as well, and then so will you. The Great Pox is also often passed down to any new children.
Also, she will have a +100 opinion bonus with her lover. If he plots to kill you, or any children you have from a previous wife, she will likely join the plot. And wives often have very high plot power.