Are 'probably' and 'actually' mutually exclusive?

Actually is used to add emphasis to the truth of what you are saying, while probably is used to say that something is likely to happen. They can certainly be used together with no contradiction.

Actually could be replaced by “as a matter of fact”.

So “they probably all are, as a matter of fact”.

However, you are unsure that they are all wrong and so it can't be a matter of fact.

This is where the confusion lies for me.

An oxymoron is a figure of speech containing contradictory terms, such as "living dead". You have not committed an oxymoron. Perhaps the word your friend was looking for is redundancy.