How did the rule of addition come to be and why does it give the correct answer when compared empirically?

I'm still a high school student and very interested in maths but none of my school books describe these kind of things, and only say how to do it not the whys. My question is very simple, for example:

        19
       +25
      = 44

because the one from adding 9 and 5 goes on to add with 1 and two. How did this rule of addition come to be?

Here's a bit of explanation that can be useful(sorry if it is frustrating): Suppose we are a 3 year old child and no one teaches us how to add and we recognize 1 is our index and 5 all palm fingers held up. Someone gives us the problem add: 1+5 so we hold 'em up, right? and again someone gives us to add 8564+2345 so we can't lift 'em up. So we try to device a rule but we don't recognize 6+4= 10 in which 0 stays and one jumps neither can we say that only the digits from rightmost positions are to be added. This is what i meant.

Well essentially what you're doing is this:

$$19+25 = (10 + 9) + (20 + 5) = \underbrace{(9 + 5)}_{\text{ones}} + \underbrace{(10 + 20)}_{\text{tens}}$$

Then you'll get:

$$14 + (10 + 20)$$

So for the ones digit you get $4$ so let's subtract that from $14$ to get $10$. For the tens digit, since you still have $10$ left over from adding the ones digits you have to "carry over" $10$ into $(10+20)$:

$$10\,\text{(carried over digit)}+(10+20) = 20+20 = 40$$

Adding the result from the ones digit gives: $$40 + 4 = 44$$

The "rule" of carrying over from one place value into another is just that, except that you do it vertically.

We start by separating the ones and the tens:

$$19 + 25 = (10 + 9) + (20 + 5)$$

We will first add the ones, then the tens:

$$(9 + 5) + (10 + 20) = 14 + (10 + 20)$$

Now we extract the ones from 14, and pass along the ten:

$$14 + (10 + 20) = 4 + 10 + (10 + 20) = 4 + (10 + 10 + 20)$$

We can now add the tens: $$4 + (10 + 10 + 20) = 4 + 40 = 44$$

Now try that for other sums, such as $29 + 43$ and $147 + 84$. In the last one, just separate the ones, the tens and the hundreds.

Every natural number $N$ in the decimal system is represented by $n + 1$ digits

$a_n \dots a_1a_0$

for which each stands for the number of powers of $10$ that add up to the given number.

$N = a_n\cdot 10^n + \dots + a_1 \cdot 10^1 + a_0 \cdot 10^0 $

Each digit $a_n$ has 10 possible "states" form $0$ to $9$ indicating how many of the powers contribute to $N$.

If you now add two numbers $A + B$

$A = a_n \cdot 10^n + \dots +a_1 \cdot 10 + a_0$

$B = b_n \cdot 10^n + \dots +b_1 \cdot 10 + b_0$

$A+B = (a_n + b_n) \cdot 10^n + \dots + (a_1 + b_1) \cdot 10 + (a_0+b_0)$

it can happen that these $(a_n + b_n)$ exceed the possible 10 "states" that are representable by 10 digits. Thus, we have to remove the exceed $c$ from $a_n + b_n$: $c = a_n + b_n - 10$ and add this exceed to the next higher power (and repeat if we get again another exceed).

Note that in other number systems (like hexadecimal) the procedure works the same except we have more (or less) states to represent with each digit.

Here's a sketch of the general theory.

Theorem: Let $b$ be an integer greater than $1$. For every positive integer $N$, there exist

1. A non-negative integer $n$,

2. Integers $a_{n}$, $a_{n-1}$, ..., $a_{2}$, $a_{1}$, $a_{0}$ satisfying $0 \leq a_{k} < b$, and $0 < a_{n}$,

such that $$ N = a_{n} b^{n} + a_{n-1} b^{n-1} + \dots + a_{1} b + a_{0} = \sum_{k=0}^{n} a_{k} b^{k}. \tag{1} $$ Moreover, the representation (1) is unique (can be done in precisely one way).

Definition: The integers $(a_{k})_{k=0}^{n}$ are called the (base $b$) digits of $N$. The string of symbols $$ a_{n}\, a_{n-1} \cdots a_{2}\, a_{1}\, a_{0} $$ is called the base-$b$ representation of $N$.

The process of carrying arises naturally when you ask how to find the base-$b$ representation of a sum $N + N'$ from the base-$b$ representations of the summands $N$ and $N'$. If \begin{alignat*}{5} N &= a_{n} b^{n} &&+ a_{n-1} b^{n-1} &&+ \dots &&+ a_{1} b &&+ a_{0} &&= \sum_{k=0}^{n} a_{k} b^{k}, \\ N' &= a_{n}' b^{n} &&+ a_{n-1}' b^{n-1} &&+ \dots &&+ a_{1}' b &&+ a_{0}' &&= \sum_{k=0}^{n} a_{k}' b^{k}, \end{alignat*} then $$ N + N' = (a_{n} + a_{n}') b^{n} + (a_{n-1} + a_{n-1}') b^{n-1} + \dots + (a_{1} + a_{1}') b + (a_{0} + a_{0}') = \sum_{k=0}^{n} (a_{k} + a_{k}') b^{k}. \tag{2} $$ If every "coefficient" $a_{k} + a_{k}'$ is smaller than $b$, then (2) immediately gives the base-$b$ representation of $N + N'$ by stringing the coefficients. The snag is, one or more coefficients $a_{k} + a_{k}'$ might be greater than or equal to $b$. In the sum $19 + 25$, for example, the "ones place" $9 + 5$ cannot be represented by a single digit. Instead, we write $9 + 5 = 14 = 10 + 4$; the $4$ stays in the ones place, but the extra $10$ carries to the tens place. Since $10$ units of one is a single unit of ten, we add one to the tens place to accommodate the carry. (The equation $9 + 5 = 14$ itself is a special case.)

Generally, if $b \leq a_{k} + a_{k}'$, write $$ (a_{k} + a_{k}') b^{k} = \bigl[b + (a_{k} + a_{k}' - b)\bigr] b^{k} = b^{k+1} + (a_{k} + a_{k}' - b) b^{k}. \tag{3} $$ Each of $a_{k}$ and $a_{k}'$ is less than $b$, so $0 \leq a_{k} + a_{k}' - b < b$. In words, $a_{k} + a_{k}' - b$ is a base-$b$ digit. Equation (3) therefore determines the carrying rule: Add the $b^{k}$ coefficients, $a_{k} + a_{k}'$. If the sum of the coefficients is greater than or equal to $b$, subtract $b$ from their sum (obtaining a proper base-$b$ digit) and add $1$ (i.e., $b$ units of $b^{k}$) to the $b^{k+1}$ coefficient, the next column to the left.

To get a clean algorithm for addition, start at the rightmost digit and work leftward, carrying as necessary. (Carrying only affects digits farther to the left. For example, a carry in the hundreds place has no effect on the tens and ones.)

What's the meaning of $19$ ? Obviously not $1+9$, but $10+9$, because the first digit "weights" $10$. Similarly, $25$ is $20+5$. The $2$ is worth $20=2\times10$. Another digit on the left would weight $100$, like $625=6\times100+2\times10+5$.

Now how do we add two decimal numbers ?

$$19+25=(1\times10+9)+(2\times10+5)$$

We can regroup $(1+2)\times 10+(9+5)=3\times10+14$. Fine for the first digit, but no so for the units: $14$ isn't a single digit. To workaround, we decompose $14=1\times10+4$.

Now, $$19+25=(1+2+1)\times10+(4)=44.$$

You can easily generalize and imagine the graphical layout that makes this process well-organized.