Loss of significance in $a-b$
When we have two given numbers $a$ and $b$ ,if $a$ is really close to $b$,when performing $ a - b$ , we lose many significant figures,and the relative error gets really big.
But why care about the relative error if the absolute error $e(a+b)$ is still less than $e(a)+e(b)$, so why do we say: "The error is large" even though it hasn't changed(?)
Or we say " We lose many significant figures", but who cares if we're within an acceptable range from that number?
This confuses me, and every article I see repeats the same thing,"We lose significant figures", But Why is that important?
To quote Wikipedia: "The effect is that the number of significant digits in the result is reduced unacceptably"Why is it unacceptable?
Note: The majority of this text is extracted from my answer to a related question.
We consider general problem of subtracting two real numbers, i.e. $d = a - b$. This problem is ill conditioned, when $a \approx b$. In particular, if $\hat{a}$ and $\hat{b}$ are approximations of $a$ and $b$ respectively, i.e., $$ \hat{a} = a(1+\Theta_a), \quad \hat{b} = b(1 + \Theta_b)$$ and $\hat{d} = \hat{a} - \hat{b}$, then $$ \frac{d - \hat{d}}{d} = \frac{b\Theta_b - a \Theta_a}{a-b}.$$ It follows that the relative error is bounded by $$ \left| \frac{d - \hat{d}}{d} \right| \leq \frac{|a|+|b|}{|a-b|}\max\{|\Theta_a|, |\Theta_b| \}.$$ If $a \approx b$, then the right hand side can be large and there is no guarantee that $d$ is computed with a small relative error. Frequently, but not universally, $d$ will be computed with a large relative error. This is the phenomenon known as subtractive cancellation or catastrophic cancellation.
If the machine uses floating point arithmetic, then the very best we can hope for is to obtain the floating point representation of $a$ and $b$, i.e., $$\hat{a} = \text{fl}(a), \quad \hat{b} = \text{fl}(b).$$ In this case $$\max\{|\Theta_a|, |\Theta_b| \} \leq u,$$ where $u$ is the unit roundoff. In IEEE single precision arithmetic $u = 2^{-24}$. In IEEE double precision arithmetic $u=2^{-53}$.
On the other hand, if $|a| \ge 2|b|$ or if $|b| \ge 2|a|$, then $$ \frac{|a|+|b|}{|a-b|} \leq \frac{3}{2}.$$ This is an application of the triangle inequality. It follows that any subtraction $d = a - b$ causes at most a modest increase in the relative error if one operand is at least twice as big as the other.