When $\Big[ uv \Big]_{x\,:=\,0}^{x\,:=\,1}$ and $\int_{x\,:=\,0}^{x\,:=\,1} v\,du$ are infinite but $\int_{x\,:=\,0}^{x\,:=\,1}u\,dv$ is finite

I have encountered a simple problem in probability where I would not have expected to find conditional convergence lurking about, but there it is. So I wonder:

• Is any insight about probability to be drawn from the fact that infinity minus infinity appears here?
• In particular, do the two separate terms that evaluate to infinity have some probabilistic interpretation?
• Is any insight about analysis or anything else to be found here?
• Generally in what situations does one write $\displaystyle \int_{x\,:=\,0}^{x\,:=\,1} u\,dv = \Big[ uv \Big]_{x\,:=\,0}^{x\,:=\,1} - \int_{x\,:=\,0}^{x\,:=\,1} v\,du$ and find that those last two terms are both infinite even though the first one converges absolutely?
• In particular, are any instances of this particularly notable or worth knowing about?

The probability problem

Suppose $X_1,\ldots,X_6$ are independent random variables each having the same exponential distribution with expected value $\mu,$ so that $$ \Pr( X_1 > x) = e^{-x/\mu} \quad\text{for } x\ge0. $$ It is desired to find this expected value $$ \operatorname E(\max\{\,X_1,\ldots,X_6\,\}) = \mu\left( 1 + \frac 1 2 + \frac 13 + \frac 1 4 + \frac 1 5 + \frac 1 6 \right) = 2.45\mu. \tag 1 $$ One fairly routine way to show this goes like this: $$ \Pr(\min\{\,X_1,\ldots,X_6\,\} > x) = \big( \Pr(X_1>x) \big)^6 = e^{-6x/\mu}\quad \text{for }x\ge0, $$ and therefore $$ \operatorname E(\min) = \frac \mu 6. $$ Let $X_{(1)}, \ldots, X_{(6)}$ be the order statistics, i.e. $X_1,\ldots,X_6$ sorted into increasing order. Then we have $$ \operatorname E(X_{(1)}) = \frac \mu 6, \quad\text{and } \operatorname E(X_{(2)} - X_{(1)}) = \frac \mu 5 $$ because that difference is the minimum of five exponentially distributed random variables. And so on through the last one.

No conditional convergence appears above.

But suppose instead we just reduce it to evaluation of an integral.

\begin{align} & \Pr(\max \le x) = \Pr(X_{(6)} \le x) = \Pr( \text{all of }X_1,\ldots,X_6 \text{ are} \le x) = \left( 1 - e^{-x/\mu} \right)^6 \text{ for } x\ge0. \\[10pt] & \text{Hence for measurable sets $A\subseteq[0,+\infty)$ we have } \Pr(\max\in A) = \int_A f(x)\, dx \\[10pt] & \text{where } f(x) = \frac d {dx} \left( 1 - e^{-x/\mu} \right)^6 = 6\left( 1- e^{-x/\mu} \right)^5 ( e^{-x/\mu}) \frac 1 \mu. \end{align}

So here's our integral: $$ \operatorname E(\max) = \int_0^\infty xf(x)\, dx. $$ No suggestion of conditional convergence, right?

\begin{align} \operatorname E(\max) = \int_0^\infty xf(x)\, dx & = \int_0^\infty x 6\left( 1- e^{-x/\mu} \right)^5 ( e^{-x/\mu}) \, \frac {dx} \mu \\[10pt] & = \mu \int_0^\infty s 6( 1-e^{-s})^5 e^{-s} \, ds \\[10pt] & = \mu \int s\, dt = \mu st - \mu\int t\,ds \\[10pt] & = \mu s(1-e^{-s})^6 - \mu \int (1-e^{-s})^6 \, ds. \end{align} Now a substitution: \begin{align} r & = 1-e^{-s} \\[6pt] s & = -\log(1-r) \\[6pt] ds & = \frac{dr}{1-r} \end{align} Our integral becomes \begin{align} & \mu ( - r^6 \log(1-r) ) - \mu \int \frac{r^6}{1-r} \, dr \\[10pt] = {} & \mu ( - r^6 \log(1-r) ) - \mu \int \left( -r^5 - r^4 - r^3 - r^2 - r - 1 + \frac 1 {1-r} \right) \, dr \end{align} Now the temptation is to write $$ \require{cancel} \xcancel{\left[ \mu \left( -r^6 \log_e(1-r) \right) \vphantom{\frac11} \right]_0^1} - \xcancel{\mu \int_0^1 \left( -r^5-r^4-r^3-r^2 - r -1 + \frac 1 {1-r} \right) \, dr }. $$ The problem is that this is infinity minus infinity, so we have conditional convergence. So suppose we write it like this: \begin{align} & \left[ \mu \left( -r^6 \log(1-r) \right) - \mu \int \left( -r^5-r^4-r^3-r^2 - r -1 + \frac 1 {1-r} \right) \, dr \right]_0^1 \\ & \text{(The above is not standard notation, as far as I know.)} \\[10pt] = {} & \mu \left[ (1-r^6) \log_e (1-r) + \left( \frac{r^6} 6 + \frac{r^5} 5 + \frac{r^4} 4 + \frac {r^3} 3 + \frac{r^2} 2 + r \right) \right]_0^1 \end{align} After we use L'Hopital's rule to evaluate the first term, this ends up being just what we see in $(1).$

Maybe I'll post my own answer if I am so inspired, but other answers may provide valuable alternative points of view. (I don't have an answer to post yet.)

Postscript:

Where I've seen something similar before is in attempts to prove that if $\Pr(X\ge0) = 1$ and $f$ is the p.d.f. and $F$ the c.d.f. of $X$, then

$$ \int_0^\infty xf(x)\, dx = \int_0^\infty (1-F(x))\,dx. $$

If you write

$$ \int(1-F(x))\,dx = \int u\,dx = xu - \int x\, du = \text{etc.,} $$

then you get infinity minus infinity. But you can do this:

\begin{align} & \int_0^\infty xf(x)\, dx = \int_0^\infty \left( \int_0^x f(x)\,dy \right) \, dx \\[10pt] = {} & \int_0^\infty \left( \int_y^\infty f(x) \,dx\right) \, dy \\[10pt] = {} & \int_0^\infty (1-F(y))\,dy. \end{align}

Tonelli's theorem is applicable since the function being integrated is everywhere non-negative, so that justifies the change in the order of integration.

Solution 1:

Let $t(s) = {\left(1-e^{-s}\right)}^6$. For each $c>0$ we have that

\begin{align} I(c):= \int_0^cs\,t'(s)\,ds \,\,&=\,\, {\Big[s\,t(s)\Big]}_{s=0}^{s=c} \,\,-\,\, \int_0^ct(s)\,ds \\\,\,&=\,\, c\,t(c) \,\,-\,\, \int_0^ct(s)\,ds. \end{align}

Let's expand $t(s)$:

$$t(s) = 1 -6e^{-s} +15e^{-2s} -20e^{-3s} +15e^{-4s} -6e^{-5s} +e^{-6s}. $$

Now, $t$ admits an indefinite integral:

$$\int\,t(s) \, ds = s - \frac{e^{-6s}}6 + \frac{6e^{-5s}}5 - \frac{15e^{-4s}}4 + \frac{20e^{-3s}}3 - \frac{15e^{-2s}}2 + 6e^{-s}. $$

It follows that we may write

\begin{align} \int_0^ct(s)\,ds &= \left[ s - \frac{e^{-6s}}6 + \frac{6e^{-5s}}5 - \frac{15e^{-4s}}4 + \frac{20e^{-3s}}3 - \frac{15e^{-2s}}2 + 6e^{-s} \right]_{s=0}^{s=c} \\&= \left( c - \frac{e^{-6c}}6 + \frac{6e^{-5c}}5 - \frac{15e^{-4c}}4 + \frac{20e^{-3c}}3 - \frac{15e^{-2c}}2 + 6e^{-c} \right) - \left( 0 -\frac16 +\frac65 -\frac{15}4 +\frac{20}3 -\frac{15}2 +6 \right) \\&= c - \frac{e^{-6c}}6 + \frac{6e^{-5c}}5 - \frac{15e^{-4c}}4 + \frac{20e^{-3c}}3 - \frac{15e^{-2c}}2 + 6e^{-c} - \frac{49}{20}. \end{align}

We hence have a more detailed expression for $I(c)$:

\begin{align} I(c) &=& &c -6c\,e^{-c} +15c\,e^{-2c} -20c\,e^{-3c} +15c\,e^{-4c} -6c\,e^{-5c} +c\,e^{-6c} \\&&-\,\,&\left( c - \frac{e^{-6c}}6 + \frac{6e^{-5c}}5 - \frac{15e^{-4c}}4 + \frac{20e^{-3c}}3 - \frac{15e^{-2c}}2 + 6e^{-c} - \frac{49}{20} \right) \\&=& &\frac{49}{20} + e^{-6c}\left(c+\frac16\right) - 6e^{-5c}\left(c+\frac15\right) + 15e^{-4c}\left(c+\frac14\right) - 20e^{-3c}\left(c+\frac13\right) + 15e^{-2c}\left(c+\frac12\right) - 6e^{-c}\left(c+1\right) \end{align}

At this point, we just let $c\to\infty$ and easily see that $I(c) \to \frac{49}{20}$.

This is a lot of 'hands on' calculation but it's basically meant to show that integration by parts on an unbounded interval is, much like the improper integral, meant to be taken as a limit. For each bounded interval $[0,c]$, integration by parts works just fine:

$$\int_0^cs\,t'(s)\,ds \,\,=\,\, {\Big[s\,t(s)\Big]}_{s=0}^{s=c} \,\,-\,\, \int_0^ct(s)\,ds $$

If we want to consider the improper integral $\int_0^\infty s\,t'(s)\,ds$, then we need to remember that by definition

$$\int_0^\infty s\,t'(s)\,ds = \lim_{c\to\infty} \int_0^cs\,t'(s)\,ds$$

and hence that

$$\int_0^\infty s\,t'(s)\,ds = \lim_{c \to \infty} \left( {\Big[s\,t(s)\Big]}_{s=0}^{s=c} \,\,-\,\, \int_0^ct(s)\,ds\right).$$

The entirety of this limit must exist (because we explicitly calculated it, or because we showed that the result is finite via other methods, like your 'probability problem' section).
However, from the existence of $\lim_{c\to\infty} f(c) + g(c)$ it does not follow that $\lim_{c\to\infty} f(c)$ and $\lim_{c\to\infty} g(c)$ both exist, and this is precisely what's going on here, with

$$f(c) = c -6c\,e^{-c} +15c\,e^{-2c} -20c\,e^{-3c} +15c\,e^{-4c} -6c\,e^{-5c} +c\,e^{-6c}$$

$$ g(c) = -\left( c - \frac{e^{-6c}}6 + \frac{6e^{-5c}}5 - \frac{15e^{-4c}}4 + \frac{20e^{-3c}}3 - \frac{15e^{-2c}}2 + 6e^{-c} - \frac{49}{20} \right). $$

If the problem is with respect to the change of variables, then there are two things to take into consideration.

The first is that if the change of variables is made after applying integration by parts, then once again we must take the limit into account. We will have

$$\lim_{c \to \infty} \left( {\Big[s\,t(s)\Big]}_{s=0}^{s=c} \,\,-\,\, \int_0^ct(s)\,ds\right)$$

and then the change of variable $r = 1-e^{-s}$ means

$$ \int_0^ct(s)\,ds = \int_0^{1-e^{-c}}\frac{r^6}{1-r}\,dr.$$

The second is that if we apply the change of variables before integration by parts... we still need to take the limit into account. That's because we had an improper integral from the start, so our calculations will look something like this:

\begin{align} \int_0^\infty s\,t'(s)\,ds &= \lim_{c\to\infty} \int_0^cs\,t'(s)\,ds \\&= \lim_{c\to\infty} \int_0^{1-e^{-c}}\,-\log(1-r)\cdot 6r^5(1-r)\,\frac{dr}{1-r} \\&= -6\lim_{c\to\infty} \int_0^{1-e^{-c}}\,\underbrace{r^5}_{u'(r)}\,\,\underbrace{\log(1-r)}_{v(r)}\,dr\tag{$1$} \\&= -6\lim_{c\to\infty} \int_0^{1-e^{-c}}\, \underbrace{r^5}_{u'(r)}\underbrace{\log(1-r)}_{v(r)}\,dr \\&= -6\lim_{c \to \infty} \left( {\left[\frac{r^6}{6}\,\log(1-r)\right]}_{r=0}^{r=1-e^{-c}} \,\,-\,\, \int_0^{1-e^{-c}}\frac{r^6}{6}\,\cdot\left(\frac{-1}{1-r}\right)\,dr\right) \\&= \lim_{c \to \infty} \left( {\Big[-r^6\,\log(1-r)\Big]}_{r=0}^{r=1-e^{-c}} \,\,-\,\, \int_0^{1-e^{-c}}\frac{r^6}{1-r}\,dr\right) \end{align}

Well, but what about $(1)$? Can't we write

$$\lim_{c\to\infty} \int_0^{1-e^{-c}}\,r^5\,\log(1-r)\,dr = \int_0^{1}\,r^5\,\log(1-r)\,dr \tag{$2$}$$

and then apply integration by parts, getting rid of the limit in the process? This would mean the problem still lies in there somewhere.

Of course, equality $(2)$ is indeed valid, but then you cannot apply IBP afterawrds. This is because the hypothesis of integration by parts would require $v(r) = \log(1-r)$ to be absolutely continuous on $[0,1]$. This clearly fails, since $v$ isn't even uniformly continuous on $[0,1)$.

On the other hand, $v$ is in fact absolutely continuous on $[0,k]$ for any $k$ with $0<k<1$. Indeed, it is differentiable and its derivative $\frac{-1}{1-r}$ is bounded on each such interval, so more strongly $v$ is Lipschitz on them. It follows that we may apply IBP inside the limit without any sort of trouble.

Solution 2:

Here is a possible reason.

$$\begin{align} \operatorname \mu^{-1}E(\max) = \mu^{-1}\int_0^\infty xf(x)\, dx & = \mu^{-1}\int_0^\infty x 6\left( 1- e^{-x/\mu} \right)^5 ( e^{-x/\mu}) \, \frac {dx} \mu \\[10pt] & = \int_0^\infty s 6( 1-e^{-s})^5 e^{-s} \, ds, \qquad s=x/\mu\\[10pt] & = \int_0^\infty s\, dt(s), \qquad t(s)=(1-e^{-s})^6\\[10pt] & = [st(s)]_{s=0}^{s=\infty} -\int_0^\infty t(s)\,ds \qquad t(s)=(1-e^{-s})^6\\[10pt] & = \lim_{s\to \infty}(s(1-e^{-s})^6)-\lim_{s\to 0}(s(1-e^{-s})^6) - \int_0^\infty (1-e^{-s})^6 \, ds. \\ &=A-B-C \end{align}\\$$

It is obvious that $$A = \lim_{s\to \infty}(s(1-e^{-s})^6)=+\infty,$$ $$B=\lim_{s\to 0}(s(1-e^{-s})^6)=0.$$ Because $\lim_{s\to\infty}e^{-s}=0$, so $$C=\int_0^\infty (1-e^{-s})^6 \, ds\text{ does not converge.}$$

So the infinity part of $A$ cancels the non convergent part of of $C$ to give you a convergent result $\frac{49}{20}$.