Why do we first introduce the open set definition for continuity instead of the neighborhood definition?

After (nearly) completing my course in topology, something weird just stuck out to me which I hadn't considered before. When first discussing continuity, we often use the following definition:

Let $X$ and $Y$ be topological spaces. We say that $f:X\to Y$ is continuous if for every open set $V\in Y$, $f^{-1}(V)$ is open in $X$.

This is a rather opaque definition and isn't quite as easily relatable to the notion we develop on $\Bbb R$ as the following definition:

Let $X$ and $Y$ be topological spaces. We say that $f:X\to Y$ is continuous if for each $x\in X$ and neighborhood $V$ containing $f(x)$, there is a neighborhood $U$ of $x$ such that $f(U)\subseteq V$.

These are of course equivalent definitions. However the latter is quite easy to connect to our normal intuition built up from real analysis: if our $x$-values are "close", then our $y$-values must be "close." Pedagogically, why have we somewhat cast away the latter definition as a mere equivalence and opted for the former? Clearly the latter is what led to the former and is, arguably, easier to latch on to. Is this somewhat of a byproduct of the category-theoretic nature of the former (with $f$ being a morphism of topological spaces) and math's general trend towards category-theoretic personifications? Can it also be attributed to early topologists wanting to separate topology from analysis in this way?

You can already see that your version involving neighborhoods is more complicated than the version involving open sets. In general, I find proving things much easier in terms of open sets, and neighborhoods are mainly for doing things that look like calculations.

An important point is that topology is more clearly expressed in terms of open sets. When you talk about topology via a basis of open neighborhoods, it obscures what the topological space actually is. If I topologize the plane where my neighborhoods are the interiors of squares, is that a different topological space than if I take the open discs as neighborhoods?

So if you define the notion of topological space in terms of open sets, you then have (at least) two choices about how to introduce continuous functions:

• Introduce them in terms of the open sets
• Introduce the notion of a basis for the topology, then define continuity in terms of a basis

I think the second choice would obscure things more. Not only do you delay introducing the notion of continuity, but the definition is also resting upon more complex ideas.

I think it is a pity that the definition of a continuous function isn't more intuitive. A lot of students have difficult to grasp topology even though it shouldn't be so and a more intuitive approach would make their studies much easier. The usual definition should be a theorem in my opinion. Given a function $f:X\to Y$, my favorite definition is:

• For all sets $B\subset Y$, it holds that $x\in \overline{f^{-1}(B)}\Rightarrow f(x)\in \overline{B}$

that tells me something like "if $x$ is close to $f^{-1}(B)$ then $f(x)$ must be close to $B$".

One can define the seemingly different concept of a neighborhood space: one defines at each point $x\in X$ a set ${\scr N}_x$ called the neighborhoods of $x$ such that ${\scr N}_x$ isa filter with respect to inclusion (it's nonempty, while $N\in{\scr N}_x$ and $N'\supseteq N$ then $N'\in {\scr N}_x$, and $N,N'\in {\scr N}_x$ then $N\cap N'\in {\scr N}_x$), and is such that for any $N\in {\scr N}_x$ there is ${\scr N}_x\ni O\subseteq N$ such that $O\in {\scr N}_y$ for every $y\in O$. You can check that the set of neighborhoods of a point (the collection of sets containing an open set that contains the point) satisfies this. Conversely, defining an open set in a neighborhood space as a set that is a neighborhood of each of its points yields a topology. It is verified this construction is bijective, so indeed one sees we can specify a topology by specifying a neighborhood system at each point. The definition of continuity is what one expects: a function $f:X \to Y$ is continuous at a point $x\in X$ iff for every neighborhood $N$ of $f(x)$, $f^{-1}(N)$ is a neighborhood of $x$.

Sometimes one simply needs to specify a neighborhood basis ${\scr B}_x$, which is slightly weaker: this is a nonempty collection of sets that contain $x$, such that $B,B'\in{\scr B}_x$, there is a third ${\scr B}_x\ni B''\subseteq B'\cap B$ (so this set is directed), and such that for any $B\in {\scr B}_x$ there is $x\in V\subseteq B$ such that $V$ contains an element of ${\scr B}_y$ for each $y\in V$. The clearest example of such a collection is the set of open balls $B(x,\varepsilon),\varepsilon >0$ in a metric space. These are called basic neighborhoods at the point $x$, and the point is that $U\in{\scr N}_x$ iff $U$ contains some $V\in {\scr B}_x$. Here one can specify possibly different systems of basic neighborhoods at a point (for example $B_n=B(x,n^{-1})$ or $V_n=B(x,n^{-2})$. For the space obtained to inherit the same topology, it is necessary and sufficient that every basic neighborhood of one collection contains a basic neighborhood of the other (much like how one checks two metrics are equivalent, i.e. that they give rise to the same topology.)

This might seem cumbersome at first, but it allows for more exotic definitions of topological spaces in (maybe?) a simpler manner: consider the closed upper half plane. Topologize it as follows: for interior points, the basic neighborhoods are the open balls lying inside the open half plane. For points on the real line, the neighborhoods are open balls tangent to the point in question, plus this point. This is a space that is separable (this is easy to check!) but is not second countable, hence it cannot be metrizable.

I think it is just useful to have alternative (equivalent) definitions for terms in topology. There are of course even more ways of characterizing continuity: $f:X\to Y$ is continuous iff $\overline {f[A]}\supseteq f[\overline A]$ for all $A\subseteq X$. It takes some arguing to show these are equivalent. I recall that the definition I just gave was particularly useful when I wanted to prove continuity of the Stone-Cech extension $\beta f:\beta X \to Y$ of a continuous function $f:X\to Y$ from $X$ into a compact Hausdorff space $Y$. The proof was almost immediate. Here are some more that come to mind:

$X$ is connected if

• there is no proper clopen subset of $X$
• $X$ is not the union of two nonempty disjoint closed sets
• $X$ is not the union of two nonempty disjoint open sets
• $X$ is not the union of two nonempty sets, neither of which contains a point or limit point of the other.

$X$ is T$_1$ if

• for any distinct $x,y\in X$ there exists an open set containing $x$ missing $y$
• every singleton is closed in $X$

It is pretty easy to show that those are equivalent. The following are not so trivial, and if you work in continuum theory you will frequently want to use different versions:

A connected compact Hausdorff space $X$ is indecomposable if

• $X$ is not the union of two proper closed connected subsets
• every proper closed connected subset of $X$ is nowhere dense
• for any nonempty open $U,V\subseteq X$, $X$ is the union of two closed sets both of which intersect $U$ and whose intersection is contained in $V$

In short, it makes life much easier when you have equivalent definitions.

Now why is the first definition of continuity you gave the standard definition? My guess is because it is the most elementary and easy to read.