What is the difference between "ostensibly" and "probably"?

If something is probably true, then it may or may not actually be true.

Something is ostensibly true if someone has told you that it is supposed to be the case.

They mean completely different things. Ostensibly means someone or something has set an expectation that a situation or condition will be a certain way; probably expresses the likelihood that it will be that way.

This book ostensibly provides the reader with all the information needed to write good prose.

Here the speaker is implying that some other information — the blurb on the book jacket, a review in a newspaper perhaps — has suggested that the book contains the information in question, but that the speaker makes no guarantees on that point and may even be setting up a rebuttal to that contention, following the first sentence with something like "But I found it an utter waste of time."

This book probably provides the reader with all the information needed to write good prose.

In this sentence the speaker believes there is a good chance that the book in question satisfies the requirements. It can also leave room open for doubt about that contention, but the opinions expressed are the speaker's and not someone else's.

You should be wary when words are used interchangably; that often means the speaker(s) do not know the precise meaning of the words.

Ostensible as an adjective means "intended for display" or "being such in appearance". As an adverb, it either carries the noun's meaning or signifies "to all outward appearances".

Thus, it should be used as a close synonym of "apparently". As Robusto noted, "probably" is a judgment, quantitative or qualitative, and should not be used as a synonym for it.