Combining two probability distributions

Solution 1:

The work by Clemen & Winkler is not for this situation. For example, if $P=(1,0,0,0)$ is a probability distribution over a $4$-element set, and $Q=(0.5, 0.3, 0.2, 0)$ is another independently obtained probability distribution over that set, then the probability distribution, $F(P,Q)$, resulting from combining information in $P$ and $Q$, should be $(1,0,0,0)$ because $P$ already has conclusive information on the set elements, that cannot be further "improved" by another observation. In other words, any $0$-value occurring in $P$ (or $Q$) must result in a $0$-value in $F(P,Q)$ at the same position (the same for any $1$-value logically follows from this). Also, the identity of $F$ should be the uniform distribution $(0.25, 0.25, 0.25, 0.25)$, as that is the most inconclusive distribution. Aggregating $P$ and $Q$ by taking their weighted arithmetic or geometric mean does not achieve this, as in most works like Clemen & Winkler. Could someone please suggest how such a function $F$ should be defined? We must assume that $P$ and $Q$ are consistent, i.e. if one of them has a $0$-value in a certain position, then the other does not have a $1$-value in the same position, because our two observations cannot contradict each other. I am not an expert in probability theory, so please pardon any inaccurate terminology usage. I would appreciate any help with finding such an $F$. Thanks!

Solution 2:

The quantity you request is the joint probability distribution $P(x_A,x_B)$, that is the probability that $A$ observes $x_A$ while at the same time $B$ observes $x_B$. You've specified that observer $A$ sees a normal distribution, in other words you're saying $$ \int P(x_A,x_B) dx_B = N(x_A;m_A,\sigma_A) = \frac{1}{\sigma_A\sqrt{2\pi}} e^{-\frac{(x_A-m_A)^2}{2\sigma_A^2}} $$ then you tell us that observer B sees a similar result with possibly different mean and sigma: $$ \int P(x_A,x_B) dx_A = N(x_B;m_B,\sigma_B) = \frac{1}{\sigma_B\sqrt{2\pi}} e^{-\frac{(x_B-m_B)^2}{2\sigma_B^2}} $$ where I have used the subscripts $A,B$ instead of $1,2$ to indicate the respective observations.

The assumption that the observations are statistically independent means that $$ P(x_A,x_B) = P(x_A)\,P(x_B), $$ which trivially satisfies the above integral relationships as $N_1,N_2$ are normalized. So to answer your initial question, the correct answer is $N_1 N_2$. One never sums probability distributions unless the observations are mutually exclusive. In other words $N_1+N_2$ could only have been the answer if when $A$ makes an observation, there is no corresponding observation from $B$. One can see there's a further problem that the sum isn't even normalized, so that could not have been the answer.

Because you specified the problem in terms of integral relationships, there are other possible answers if statistical independence is not assumed. Consider for example a multivariate normal distribution, which can satisfy the above integral relationships though it is not separable. One can further make the multivariate normal distribution elliptical, as long as the projections onto the two coordinate axes are held fixed. In other words, there could be an off-diagonal correlation matrix. One should always check the correlation matrix for real measurements, because sometimes you can have a very precise measurement in an off-diagonal direction like $x_A+x_B$ but very large errors in the orthogonal direction ($x_A-x_B$), and this would show up as mediocre errors on both axes when projected onto the coordinate axes.