Why "a function continuous at only one point" is not an oxymoron?

Sometimes, intuition is wrong, and we should learn from the math, rather than try to make the math fit the intuition.

That said, I bet what you are probably intuiting is not something that should be called "continuous at the point $P$" at all -- it should be called "continuous in a neighborhood of the point $P$". e.g. "there exists $a < P < b$ such that $f$ is continuous on $(a,b)$". So the math already fits the intuition, once you correctly translate between them.

Via nonstandard analysis, there is a "infinitesimal segment" approach to continuity-at-a-point: a function $f$ is continuous at a point $a$ if, for any infinitesimal $h$, the difference $f(a+h)-f(a)$ (actually, ${}^*f(a+h)-{}^*f(a)$, to be precise; see below) is infinitesimal. That is, $f$ is continuous at $a$ if the graph of $f$ "infinitesmially near" $a$ varies only infinitesimally from a straight (horizontal, in fact) line. This definition is equivalent to the usual $\epsilon-\delta$ definition.

OK, so what is this "nonstandard analysis" I mention? Well, this is certainly too complicated for a short paragraph, but basically the idea is this: we start with the "real" universe of the real numbers $\mathbb{R}$ and all functions (continuous or not) on $\mathbb{R}$, and we consider a "blown up" version of the reals, called $^*\mathbb{R}$. $^*\mathbb{R}$ is an ordered field which contains the real numbers, and a lot more junk besides, including field elements which are less than every (real) positive real number; we call these infinitesimals. Moreover, each function $f$ on $\mathbb{R}$ has a version $^*f$ defined on $^*\mathbb{R}$, which agrees with $f$ on the actual reals. Statements in the $\epsilon-\delta$ language can be translated to (arguably) more intuitive and snappy definitions in terms of infinitesimals.

If this all seems suspicious to you, that's very reasonable; it takes a lot of work to set this up so it doesn't break. There are many good sources with actual details; I like http://homepages.math.uic.edu/~isaac/NSA%20notes.pdf, but I'm biased (I learned from it). See especially section 2, which explains why we can get away with this sort of silliness :D.

Definition: For topological space $(X,\mathcal{T})$ and $(Y, \mathcal{S})$, a mapping $f: X\rightarrow Y$ is said to be continuous at the point $x_0$ in $X$ provided for any open neighborhood $O$ of $f(x_0)$, there is an open neighborhood $U$ for which $f(U) \subset O$.

Then we say $f$ is continuous provided it is continuous at each point in $X$.

This is the topological definition of continuity, even if you do not understand anything about topology, that is okay. The important thing to take away from this definition is that the idea of continuity at a single point comes first. Then if the function is continuous at each point in the domain, we abbreviate this by simply saying the function is continuous.