What exactly is nonstandard about Nonstandard Analysis?

The term "nonstandard" refers to nonstandard analysis using a nonstandard model. In nonstandard analysis, one usually intends to study the real numbers (and their functions and relations). The real numbers under certain operations model a field. In nonstandard analysis, the intended model consists of the real numbers, while the model used for study of the real numbers usually consists of a hyperreal field. Hyperreal fields are not isomorphic to the real number field. We can see this just by looking at finite hyperreal structures, that is just finite numbers, infinitesimals, and numbers infinitely close to finite numbers.

Given a hyperreal field with only finite elements, we do have a homomorphism from those finite hyperreals to the reals, at least once we take care of division. So, in the finite case, we can regard the "standard part" or "shadow" of a hyperreal number as a function, but we cannot correctly obtain an inverse function from the reals to the finite hyperreals. We also have a homomorphism from basically any structure of finite hyperreals to a given real structure.

For example, if we consider $(\mathbf{H}, +', *', -', /')$, that is the finite hyperreals under hyperreal addition, multiplication, subtraction, and division (where the denominator is not an infinitesimal) we have a map $\DeclareMathOperator{\SH}{SH}\SH$ to $(\mathbf{R}, +, *, -, /)$ such that

$$\begin{array}{} \SH(h&+'&i)&=&(\SH(h)&+&\SH(i))\\ \SH(h&*'&i)&=&(\SH(h)&*&\SH(i))\\ \SH(h&-'&i)&=&(\SH(h)&-&\SH(i))\\ \SH(h&/'&i)&=&(\SH(h)&/&\SH(i))\\ \end{array}$$

and for all finite hyperreal numbers $h$, there exists a real number $r$ such that $\SH(h)=r$.

No inverse of the shadow function from the reals to the finite hyperreals exists, or in other words no function $I$ exists such that $I(\SH(h))=h$ holds true. This comes perhaps as easiest to see if you consider that the shadow of each infinitesimal number equals $0$. But, $0$ does not associate with a unique hyperreal infinitesimal, since we could associate $0$ with one of an infinity of positive infinitesimals, or one of an infinity of negative infinitesimals in rather the same way.

See Pete L. Clark's last two comments below also.

Also, the Dedekind completeness property fails for the hyperreals, which indicates the hyperreals as a non-isomorphic structure to the reals under various operations. Keisler's text referenced by Brian M. Scott makes for a good reference to read, and it has some good examples.

It might help to understand a tiny little bit of the background leading Abraham Robinson to developing his model of nonstandard analysis. There have been several approaches to incorporating infinitesimals into analysis. One such approach was developed by Schmieden and Laugwitz. They considered sequences of real numbers under the equivalence relation where two sequences are equal if from some point onwards all their coordinates agree. This leads to an extension of the real numbers but does not yield a field (but rather a ring with zero divisors). Nonetheless, one can still do quite a lot of analysis in this larger ring which includes proper infinitesimals.

Roughly the same time Skolem had used fundamental results in logic (model theory) to construct models of the natural numbers that included infinitely large natural numbers. These models were not intended to have any applications but rather used for logic oriented investigations. Skolem called these models of the naturals nonstandard models of arithmetic.

Robinson was well aware of the work of Schmieden and Laugwitz and of Skolem's work. Robinson's great contribution was in the realization that the Schmieden-Laugwitz model can be greatly improved utilizing Skolem's techniques. It is thus almost certain that Robinson chose the term nonstandard analysis due to the existing nonstandard models of arithmetic.

To complement the fine answers given earlier, I would like to address directly the question of the title: "What exactly is nonstandard about Nonstandard Analysis?"

The answer is: "Nothing" (the name "nonstandard analysis" is merely a descriptive title of a field of research, chosen by Robinson). This is why some scholars try to avoid using the term in their publications, preferring to speak of "infinitesimals" or "analysis over the hyperreals", as for example in the following popular books:

Goldblatt, Robert, Lectures on the hyperreals. Graduate Texts in Mathematics, 188. Springer-Verlag, New York, 1998

Vakil, Nader, Real analysis through modern infinitesimals. Encyclopedia of Mathematics and its Applications, 140. Cambridge University Press, Cambridge, 2011.

More specifically, there is nothing "nonstandard" about Robinson's theory in the sense that he is working in a classical framework that a majority of mathematicians work in today, namely the Zermelo-Fraenkel set theory, and relying on classical logic.