When solving an overdetermined linear system, is it possible to weight the influence of each equation?

Currently, I am calculating the pseudo-inverse to solve the system, which is simple and fast for my purpose (with a computer, obviously). However, each equation represents an individual from a population. For reasons outside the scope of this site, I need to give more importance to some individuals than others.

E.g., consider a total of $N$ individuals. $0.1N$ would be special and have weight $2$. The only way I can think of to accomplish that is to duplicate the $0.1N$ corresponding equations. Since I am no math expert, I don't know if it is effective or with no effect at all. If equation duplication is effective, ok, that's a solution, but with a minor problem: it does not allow to use non-integer weights.

obs.: I don't know for sure, but the answer for non-overdetermined linear systems might also apply.


If you have an overdetermined system of linear equations

$$Ax=b$$

you can weigh the error with a diagonal weight matrix $W$

$$e=W(Ax-b)$$

This matrix assigns different weights to each equation. Now you can solve the problem by minimizing

$e^Te=(Ax-b)^TW^TW(Ax-b)$

which gives you a system of linear equations for $x$:

$$A^TW^TWAx=A^TW^TWb$$

This system can be solved for $x$ assuming matrix $A$ has full rank.


You can use a linear or non-linear fitting package. Normally these have a way to specify the measurement error of each point. Points with lower measurement error are weighted more heavily. A discussion is in chapter 15 of Numerical Recipes as well as most numerical analysis texts.