Does my opinion of other people matter?

It has an impact when the person is your liege. Negative opinion means your liege gets less taxes out of your demesne and less troops if (s)he should decide to raise your levies. Other than that, the opinion modifier is mostly used by the AI.

Also, from what I understand, your opinion of your spouse does not impact fertility.

(As an aside, your own opinion is a handy indicator of a vassal being in a faction conspiring against you. Mouse over your opinion of a character and if you see a -10 modifier "Faction member" you know they are up to no good. At this point use standard tricks to get them out of the faction before it becomes too powerful.)

Unfortunately, I don't think there's been any documentation of this having a particular effect. However, I think it probably works in a lot of the ways it works for you interacting with NPCs. The most evident is that the NPCs are not likely to ask you to join a plot to murder someone that your character likes, for instance.

I think the primary (only?) use for your personal opinion, is for roleplaying purposes. For example, if your character really loathes his wife, he might want to divorce her or have an affair – and if he has a high intrigue rating, it might be in his character to see her dead.

Not all players choose to roleplay, but for some it adds a bit more flavour to the game.