What is the intuition for the point-set topology definition of continuity?

Let $X$ and $Y$ be topological spaces. A function $f: X \rightarrow Y$ is defined as continuous if for each open set $U \subset Y$, $f^{-1}(U)$ is open in $X$. This definition makes sense to me when $X$ and $Y$ are metric spaces- it is equivalent to the usual $\epsilon-\delta$ definition. But why is this a good definition when $X$ and $Y$ are not metric spaces? How should we think about this definition intuitively?

One abstract way to think about continuity (in the sense that it generalizes to non-metric spaces) is that it is about error. A function $f : X \to Y$ is continuous at $x$ precisely when $f(x)$ can be "effectively measured" in the sense that, by measuring $x$ closely enough, we can measure $f(x)$ to any desired precision. (In other words, the error in our measurement of $f(x)$ can be controlled. "Precision" here means "to within an arbitrary neighborhood of $f(x)$," so it does not depend on any metric notions.) This is an abstract formulation of one of the most basic assumptions of science: that (most of) the quantities we try to measure ($f(x)$) depend continuously on the parameters of our experiments ($x$). If they didn't, science would be effectively impossible.

If you like thinking about limits, a function is continuous if and only if it preserves limits of filters or, equivalently, nets. These are two ways to generalize converge of sequences to spaces which are not first-countable.

Maybe it's just me, but I've never thought that the usual $\epsilon$-$\delta$ definition of continuity is intuitive at all. Why should a function be continuous at $x$ if every ball of radius $\epsilon$ around $f(x)$ contains the image under $f$ of a ball of radius $\delta$ around $x$?

Instead, in metric spaces, I think of a function as continuous if it preserves limits, which can be intuitively (and generalizably) be phrased by saying that $f$ is continuous if and only if whenever $x$ is in the closure of a set $A$, then $f(x)$ is in the closure of the set $f(A)$.

(Take a piece of paper and draw out the arguments of the next two paragraphs)

To see that the $\epsilon$-$\delta$ continuity implies 'closure' continuity, suppose that $f$ is not closure continuous at $x$, that is $x$ is in the closure of some set $A$ but $f(x)$ is not in the closure of $f(A)$. Then there exists an $\epsilon$-ball around $f(x)$ that does not intersect $f(A)$ even though every $\delta$-ball intersects $A$. Hence, some $\epsilon$-ball around $f(x)$ contains no image of a $\delta$-ball around $x$, and so $f$ is also not $\epsilon$-$\delta$ continuous.

To see that 'closure' continuity implies $\epsilon$-$\delta$ continuity, suppose that $f$ is not $\epsilon$-$\delta$ continuous. Then there exists an $\epsilon$-ball around $f(x)$ that contains no image of a $\delta$-ball around $x$. In other words, the preimage of the $\epsilon$-ball around $f(x)$ contains no $\delta$-ball around $x$, so let $A$ be the collection of points that are not in the preimage of the $\epsilon$-ball. Then $x$ is in the closure of $A$ since any $\delta$-ball around $x$ has a point outside the preimage of the $\epsilon$-ball and hence in $A$, but $f(x)$ is not in the closure of $f(A)$ since the $\epsilon$-ball around $f(x)$ is disjoint from $f(A)$.

Now, the cool thing to notice is that the above equivalence of definitions works perfectly fine if you replace $\delta$-balls and $\epsilon$-balls with open sets in the appropriate topological spaces, so really what you should care about is how to make sense of 'closure' continuity in a space that is not a metric space, and the answer is given by the wiki:Kuratowski Closure Axioms.

You might also find useful the answers to this mathoverflow question, specifically this one by sigfpe and this one by Vectornaught. The first one talks about how open sets can be thought of as rulers that try to measure imprecisely things in the vector space (but which doesn't explain why continuity it as it is)), while the second phrases the Kuratwoski Closure Axioms in terms of the intuitive notion of 'nearness' of points (which does account for continuity).

Intuitively:

Continuous maps are exactly those maps that preserve (in the forward direction) the notion of "closeness": A map $f : X \to Y$ is continuous iff points "close" to each other in $X$ are always sent to points that are (once again) "close" to each other in $Y.$

I now explain what this means exactly and why the most intuitive way of thinking of continuity is actually through its characterization in terms of closed sets${}^{1}$. In short, it allows you define continuous maps as being exactly those maps that preserve a certain property in the forward direction.${}^{2}$ Let me introduce some non-standard (i.e. my own made up, but sensible) definitions:

1. Say that a point $y$ is close to a set $S$ if $y \in \overline{S}.$
2. Say that a set $R$ is close to a set $S$ if $R \subseteq \overline{S}$ (i.e. if $R$ is contained in the closure of $S$).

With these definitions, a subset is closed if and only if it contains every point/subset that is close to it (so the terminology "being close" intuitively describes "being closed"). Recall how the closure operator characterizes continuity:

Theorem (non-intuitive statement): A map $f : X \to Y$ is continuous if and only if for all subsets $A \subseteq X$, $f\left( \overline{A} \right) \subseteq \overline{f(A)}$.

This can be restated as:

Theorem (intuitive statement): A map $f : X \to Y$ is continuous if and only if for all subsets $A \subseteq X$, $f$ maps points that are close to $A$ to points that are close to $f(A)$.

You can replace the word "points" above with the word "sets" and the resulting statement will still be true. Thus continuous maps are exactly those that preserve (in the forward direction) the notion of "closeness" in $X$.

If this interpretation is valid then you might expect for the following characterization to also be valid.

Continuity at a given point $x \in X$: $f$ is continuous at $x$ if and only if whenever $x$ is close to a subset $A \subseteq X,$ then $f(x)$ is close to $f(A).$

You can check that this is characterization of continuity at a point does actually hold. It also follows immediately from the above two characterizations that $f$ is continuous if and only if it is continuous at every point of its domain.

1. You can actually (essentially) define the category of topological spaces using only the closure operator: see Kuratowski closure axioms (technically, this category is equivalent to the category of topological spaces). This justifies thinking of topological spaces in terms of "closeness" rather than open subsets.

2. This is in contrast to the open set definition of continuity, which defines continuous maps as being those that preserve a certain property (i.e. openness of subsets) in the backwards direction.

You have it exactly right. It works well when $X$ and $Y$ are metric spaces, and has proved useful in more general contexts. When you think of open sets as points "near" another, this is the proper translation of the usual $\epsilon-\delta$ definition.