Are all "numbers" just one unit value transformed by a function?

I'm a programmer, and I've been thinking about this for some time. I have a saying:

There are only three specific numbers: 0, 1, and ∞. ∞ is just a special case of 1.

Our base function really just converts a unary value into a string (example: "101") that we can parse in a given base.

Tonight, I've been putting some thought towards this, and I realize - I believe that the entire set of natural numbers can be represented by a recursive function that only works on a unit value, as follows (I apologize in advance for the terrible notation, this is not my forte):

// Please ignore the fact that this is invalid code for various, various reasons
N (the set of natural numbers) = add(1) where add(x) = return x + add(x);

Following this, I believe that the set of integers can be found as a union of the natural number function and a similar function using subtract instead of add (resulting in the full set of negative numbers).

Going further, I applied more thought - perhaps the set of real numbers can be applied through division functions for the set of all integers divided by the set of all integers (which we've already established via the functions above).

I suppose my question is this - as strange as it may seem, is there only one unit value? (Or, possibly, a very small subset of seed numbers - perhaps 0 and 1 (and maybe -1 for the complex plane?)) Is our number system just a system of transforms on top of one unit value?

If this is a really stupid question, I apologize. It's just that the more I think about this, the more I really wonder about it. There's a good chance that I'm missing something or that this has already been derived, but I'd love to know. I'm sure this community knows much more than I do about this. I hope that this interests all of you as much as it does me.

the entire set of natural numbers can be represented by a recursive function that only works on a single unary value

Bravo -- this is exactly the key insight that is exploited in most standard formalizations of the natural numbers (the most famous of which is called Peano arithmetic, if you are curious about the details). The recursive function you refer to, which you denoted add(1), is usually called the successor function and written $S$. It's a function from the natural numbers to the natural numbers, which takes a natural number $n$ and returns $n+1$.

To define the natural numbers $\mathbb{N}$, we say that $0$ is a natural number, and for any natural number $n$, $Sn$ is a natural number. This gives us all natural numbers: $0$, $1 = S0$, $2 = SS0$, $3 = SSS0$, and so on. We also may want to say that $Sn$ is a new natural number, one that has not been defined before (so for example, $4 = SSSS0$ is not equal to $2 = SS0$).

Some logics are strong enough that this definition alone is enough to define and prove what we want about the natural numbers! You seem to be interested in formalizing it as a program. In the proof assistant Coq, we may define the natural numbers as:

Inductive naturals : Type :=
  | O : naturals
  | S : naturals -> naturals.

This is similar to your code snippet, but unlike yours (which was invalid code) this is perfectly valid code defining the naturals. Yes, it really is that simple -- nothing else is needed. Even more surprising, we can go from here to define addition, multiplication, exponentiation, prime numbers, and whatever else we like, and prove things about these that we want to be true. (Most of the definitions will be recursive, and the proofs will be by induction.) Once we have the successor function $S$, everything else follows.

Following this, I believe that the set of integers can be found as a union of the natural number function and a similar function using subtract instead of add (resulting in the full set of negative numbers).

Yes, this basic idea will work to define the integers $\mathbb{Z}$ from $\mathbb{N}$. The integers could be defined by $0$, the successor function $S$, and a predecessor function $P$ which takes $n$ to $n-1$. Unlike with the natural numbers, we do have to define a special equality (equivalence relation) on the integers, so that $PSn$ and $SPn$ are both equivalent to $n$.

An alternate definition (perhaps more common) is that an integer is an ordered pair $(a,b)$, where $a$ and $b$ are natural numbers, and this represents informally the integer $a-b$. We then have to define what it means for two integers to be equal (an equivalence relation), and we say that $(a,b) = (c,d)$ if $a + d = b + c$.

Once again, either definition may be easily formalized in a programming language like Coq.

Going further, I applied more thought - perhaps the set of real numbers can be applied through division functions for the set of all integers divided by the set of all integers (which we've already established via the functions above).

Introducing division in this way (fractions) will get you from $\mathbb{Z}$ to the rational numbers $\mathbb{Q}$. But it won't get you the real numbers $\mathbb{R}$, yet, which is a MUCH bigger set than $\mathbb{Q}$. There are a lot of real numbers -- so many that computationally speaking, we can't really write down most of them. As a theoretical tool, however, it's useful to define this giant set, which we do usually using "Dedekind cuts" or "Cauchy sequences". There are several other less-known ways as well. You can read about it here.

I suppose my question is this - as strange as it may seem, is there only one unary value? (Or, possibly, a very small subset of seed numbers - perhaps 0 and 1 (and maybe -1 for the complex plane?)) Is our number system just a system of transforms on top of one unary value?

Yes, the idea is that we can start with $0$ as the only base value, and use the successor function $S$, together with a way of constructing ordered pairs (for subtraction and division) to generate all the numbers we need. More generically, we start with one or several base values (seed numbers) and one or several recursive constructors (transforms) to build up all possible numbers.

We can develop the complex numbers $\mathbb{C}$ from $\mathbb{R}$ using ordered pairs again where $(a,b)$ is $a + bi$.

However, we do have to be careful to understand the limits of our construction. For example, for lots of computations involving natural numbers to be efficient, it is actually essential that they be represented as binary strings and not unary strings of $S$s. So if we care about efficiency, then not all representations are created equal.

You may be interested in reading about groups, rings, and fields. Wikipedia is a good place to start.

We say that 1 generates all integers under addition and inversion, that is, every positive integer is just a sum of ones, and and every negative integer is just its negation.

If we introduce division, then we get all rational numbers. If we further allow roots of polynomials we get algebraic numbers which include some complex numbers. However, we cannot get all real numbers using algebraic techniques like this--such numbers are called transcendental numbers, e.g. $\pi$ and $e$.

This is a really interesting part of math and not a stupid question at all!