"Information is a distinction that makes a difference" - Donald MacKay

Solution 1:

A linguistic example could be any classic minimal pair, like PIN/PEN. This is a distinction (/I/ vs. /E/), which makes a difference for the hearer: "Please hand me that pin." Because both 'pin' and 'pen' are valid options in that sentence, the way to get the correct message is to use that distinction.

This distinction could also fail to make a difference. If, for example, the word 'pin' did not exist (indeed, if English lacked the /I/-/E/ distinction entirely), then "Please hand me that pen." could be safely understood to refer to the object 'pin'. (This simplifies a little and assumes other minimal pairs aren't important.) In this case, the acoustic distinction is not missing, but it doesn't make a difference.

This concept is usually treated more formally in information theory, which might be a fruitful place to read more.

Solution 2:

This phrase is attributed to Donald Mackay but it doesn't seem to be possible to identify a source.

"A difference that makes a difference" on the other hand, is discussed at length by Gregory Bateson, as a definition of Information:

"The technical term "information" may be succinctly de-fined as any difference which makes a difference in some later event. This definition is fundamental for all analysis of cybernetic systems and organization. The definition links such analysis to the rest of science, where the causes of events are commonly not differences but forces, impacts, and the like. The link is classically exemplified by the heat engine, where available energy (i.e., negative entropy) is a function of a difference between two temperatures. In this classical instance, "information" and "negative entropy" overlap. Gregory Bateson Steps to an Ecology of Mind.Collected essays in anthropology, psychiatry, evolution, and epistemology more details