What is a big condition number for a matrix?

The condition number of a matrix is a measure of how close a matrix is to being singular.

But, what is considered a big condition number?

Solution 1:

You can view a solver as a black box which maps the input (in your case, the right-hand side vector) into the output (in your case the solution). The condition number connects the relative error of the input to the relative error of the output. Ideally, we have a relationship of the form $$ \text{relative error on the output} \leq \text{condition number} \times \text{relative error on the input}.$$ This assumes well written code with no rounding errors made during the computation. While this is nearly impossible to achieve in practice, relations of this type set clear limits for what can be achieved. If we need a relative error of $10^{-6}$ on the output side and we have a relative error of $10^{-16}$ on the input side, then a condition number of $10^{10}$ is the largest value we can tolerate. Anything larger is too large, anything smaller is just a bonus.

Therefore, when classifying a condition number as large or small it is not enough to simply look at the number. We must have knowledge of the application: what sort of accuracy is required/available. It is also useful to know the value of the unit roundoff on the machine in question, because this is a good indicator of the smallest relative error we can expect on the input side.