Hyphens after the prefixes “non-” and “anti-” in mathematics
As with most issues involving English spelling, there is no right or wrong here, only preferences that vary substantially by region, by publisher, and by writer, so much so that is easy, maybe even trivial, to find living counterexamples of any posited general rule here.
But in general, British publishers tend to be more tolerant of the hyphen than American publishers tend to be. Many American publishers forbid the hyphen except when the word following is capitalized, as in anti-American or anti-Semite, or when you would form two vowels in collision, as in anti-immigration. (You never seem to see antiïmmigration, though; it almost looks like someone has smudged the page in this font.)
Otherwise American publishers tend to prefer versions like nonnative, nonnaturalized, nonzero, antilogarithm, antiperiodic, antisocial, antitrust. However, with individual writers not forced to conform to any particular style guide, usage varies.
As tchrist says, there's a lot of variation. I think in general:
We tend to use a hyphen after "anti" when what follows begins with a vowel, especially an "i", because without it the word can be difficult to read. "Antisocial" is easy to read; "antiimigration" (to take tchrist's example) is awkward. (Similarly for other prefixes that end with a vowel, like "pre": we tend to write, for example, "premarital agreement", but "pre-employment agreement". Probably because "preemployment" looks like it should be read with "preem" as the first syllable, like preem-ploy-ment.)
Words indicating opposition to some social or political idea tend to be written with a hyphen. That is, when there's a "pro" and an "anti" we write both with hyphens. Like "pro-gun" and "anti-gun" for the sides in the second amendment/gun control debate, "pro-Israel" and "anti-Israel" for sides in debates about that nation, etc. But words where there's not a "pro-X", more just "X" and "anti-X", tend to not use a hyphen, like "logarithm" and "antilogarithm".
I don't have documentation on this but I suspect that newly-coined words tend to use the hyphen, but it gets dropped as the word becomes common or accepted.