Motivation for the concept of "open set" in topology

You could just as well define your topology by defining the closed sets, demanding that your space and the empty set be closed, and that arbitrary intersections and finite unions of closed sets are closed. Since open sets are the complements of closed sets, this would then give us the open sets, and define the exact same topology.

For a more pedagogic or philosophical answer, see this mathoverflow thread for some good discussion on the motivation/interpretation of open sets: https://mathoverflow.net/questions/19152/why-is-a-topology-made-up-of-open-sets


Since closed sets are, by definition, exactly those whose complements are open, we could do everything just as well by speaking about closed sets instead of open ones. The fact that open sets are the most common way to define a topology is purely conventional.


Your intuition about the "fuzziness" of open sets is correct.

Inuitively, if $U$ is an open set in the topological space $X$ and $x \in U$, then $U$ necessarily contains any other point in $X$ that is "near enough" to $x$.

Now this statement has a precise meaning in a metric space: in that setting, "near enough" means "within distance $\epsilon$" for some $\epsilon > 0$", and then the preceding statement becomes the definition of an open subset of a metric space. In a general topological space, open sets are what provide the sense of "nearness", so the preceding statement is circular at best, but it is still what you should keep in mind. You should contrast it with closed sets: if $x$ is a point on the boundary of a closed set $F$, then there will be points in $X$ that come as close as you like to $x$ but are not in $F$ (the points on the "other side of the boundary" to $F$, intuitively speaking: think of the end-points of a closed interval, or a point on the boundary of a closed disk).

The preceding discussion is purely intuitive: to really see how the definitions of closed and open sets are used you have to get a little further in your study of topology and see the technical way in which these notions are used. If you haven't already done so, it can be good to start with the metric space case: then the open and closed subsets are defined in terms of the metric, as is the concept "sufficiently close to $x$", and the above intuitions become precise.
Then, in your study, you will see that lots of concepts can be defined, and that lots of theorems can be proved, purely in terms of open and closed subsets, without having to explicitly refer to the metric. (E.g. the definitions and basic properties of continuity, convergence, compactness, connectedness, etc.) At this point it will seem natural to abstract out the properties of open and closed sets, without using the crutch of a metric to define them.