"Semitic" and "anti-Semitic"

NOAD defines anti-Semitism thus:

anti-Semitism
hostility or prejudice against Jews

And here is the relevant entry from the Online Etymology Dictionary:

anti-Semitism
also antisemitism, 1881, from Ger. Antisemitismus, first used by Wilhelm Marr (1819–1904) German radical, nationalist and race-agitator, who founded the Antisemiten-Liga in 1879; see anti- + Semite. Not etymologically restricted to anti-Jewish theories, actions, or policies, but almost always used in this sense. Those who object to the inaccuracy of the term might try H. Adler's Judaeophobia (1882). Anti-Semitic (also antisemitic) and anti-Semite (also antisemite) also are from 1881, like anti-Semitism they appear first in English in an article in the "Athenaeum" of Sept. 31, in reference to German literature.

Anti-Semitic is Jewish-specific for historical reasons, as revealed by the Etymology Dictionary. In its most literal sense, anti-Semitic should relate to all Semitic cultures, but this is not the case—a great example of how history, politics, etc, shape English usage.

Because when the word was coined and came into use, Jews were the only Semitic people encountered in modern European (and American) society.