Why are so many of the oldest unsolved problems in mathematics about number theory?

Stillwell mentions in his book, Mathematics and its History that:

Most of the really old unsolved problems in mathematics, in fact, are simple questions about the natural numbers...

Have people tried to understand what it is about the "topology" (figuratively speaking) of number theory that makes open problems from it notoriously difficult to solve?

I don't think the questions of number theory are necessarily more difficult to answer. Rather, they're easier to ask. Integers are familiar objects and they are easy to play with, and for this reason more questions have likely been asked about them throughout history than about any other objects in mathematics. Once in a while, one of these problems resists all attempts...

That being said, some problems in number theory are incredibly difficult to solve, but incredibly easy to state. How many solutions in integers does $x^{17}+y^{17} = z^{17}$ have? Given a mathematical question, one could try to quantify the ratio of "how difficult it is to solve" to "how difficult it is to state" (perhaps making use of the length of the shortest proof in a given formal language). I wouldn't be surprised if most of the mathematical questions which maximized this ratio in a suitable sense were elementary questions about the integers. Why? I guess it's kind of a philosophical question. Perhaps the integers are the first "highly nontrivial" objects in mathematics, should the objects of mathematics be ordered according to the length of their definition. But really, I have no idea.

[The question in the body of the OP is a bit different from the question in the title, and seems more interesting to me. So that is the question I am answering.]

It is common to focus on the fact that it is easy to state number-theoretical problems, since the basic concepts of number theory are fairly elementary. This is certainly true, but it I think it is worth noting another aspect of number theory which can make its problems difficult, which is that the elementary nature of the phrasing of certain problems can belie the hidden structures that underly their deeper nature and (eventually) their solution.

E.g. consider the problem of writing primes in the form $x^2 + n y^2$. When $n = 1$, there is an obvious congruence obstruction mod $4$ to solving $p = x^2 + y^2$, which implies that if $p$ is odd, then in fact $p \equiv 1 \bmod 4$. Fermat showed that this necessary condition is also sufficient.

Similarly, if we want to write $p = x^2 + 5y^2$, then if $p \neq 2, 5$ there is are obvious obstructions: $p$ should be a square mod $5$, and should be a sum of squares mod $4$, and hence $1 \bmod 4$. Together these imply that $p \equiv 1$ or $9 \bmod 20$. Again, Fermat (I believe) proved that this necessary condition is sufficient.

Now consider writing $p = x^2 + 23y^2$. There is no obstruction mod $4$, so the only obvious obstruction is that $p$ should be a square mod $23$. However, this necessary condition is no longer sufficient. E.g. $p = 13$ is a square mod $23$, but can't be written in the form $x^2 + 23y^2$, and the same is true for $p = 29$. (In fact $p = 59$ is the first prime that can be so written.)

Understanding this phenomenon led to many deep discoveries of structure in number theory, such as quadratic reciprocity, and Gauss's theory of quadratic forms. These in turn were major contributors to the development of algebraic number theory and class field theory. From a modern point of view the difference between the case $n = 23$ and the others I considered is that the class group of $\mathbb Z[(1+\sqrt{-23})/2]$ is not a product of cyclic groups of order $2$ --- it is in fact a group of order $3$.

Another example is given by formulas for sums of squares.

There is a formula for the number of ways to write a number as the sum of two squares: $$r_2(n) = 4\sum_{d |n} \chi(d),$$ where $\chi(d) = \pm 1$ if $d \equiv \pm 1 \bmod 4,$ and $0$ if $d$ is even.

There is also a formula for the number of ways to write a number as the sum of four squares: $$r_2(n) = 8 \sum_{d | n, 4\not\mid n} d.$$

There are similar formulas for sums of six or eight squares, but not for higher numbers of squares. It is hard to understand this without realizing that these formulas are governed by the structure of certain spaces of modular forms, which contain no cuspforms in low weights, but which eventually contain cuspforms, for which no elementary formulas are available.

There is an enormous amount of structure present in number theory, despite the elementary manner in which some its problems can be phrased. I believe this is one of the reasons its problems can be so hard to solve; because their phrasing often doesn't make this structure apparent, but finding it is necessary to solve the problems. (The Riemann hypothesis can be thought of as one current illustration of this difficulty.)