Log likelihood of a realization of a Poisson process?

For an inhomogeneous Poisson process with instantaneous rate $\lambda(t)$, the log likelihood of observing events at times $t_1,\ldots,t_n$ in the time interval $[0,T)$ is given by

$ \sum_i \mathrm{log}\lambda(t_i) - \int_0^T \lambda(t) dt$

I am told this can be derived by taking the limit of the discrete-time case as the bin width $\Delta t$ goes to $0$:

$ \sum_i \mathrm{log}(\lambda(t_i)\Delta t) + \sum_{t\notin \{t_1,\ldots, t_n\}} \mathrm{log}(1-\lambda(t) \Delta t)$

For the second term, it is clear how this limit works if we take the Taylor expansion:

$\mathrm{log}(1-\epsilon) \approx -\epsilon$

However for the first term, it is not clear to me how one goes from $\sum_i \mathrm{log}\lambda(t_i) + \mathrm{log} \Delta t$ to $\sum_i\mathrm{log}\lambda(t_i)$ as $\Delta t \rightarrow 0$. Shouldn't the $\mathrm{log}\Delta t$ terms go to $-\infty$?

Edit

I think I understand now where my confusion was coming from. If we look at the probability, instead of log probability, then as $\Delta t$ becomes small the probability approaches:

$ \Delta t^n \prod_{i=1}^n \lambda(t_i) \mathrm{exp}\left(-\int_0^T\lambda(t)dt\right)$

The probability of an event happening in the interval:

$[t_1 - \Delta t/2, t_1 + \Delta t/2] \times [t_2 - \Delta t/2, t_2 + \Delta t/2] \times \ldots \times [t_n - \Delta t/2, t_n + \Delta t/2]$

should scale as $\Delta t^n$ for small $\Delta t$, so while the probability of events happening at exactly those times goes to zero (as it must), the density does not.

Solution 1:

Disclaimer: This is, so far, one of my most downvoted answers on the site. Needless to say, it is perfectly correct, and it answers the question as formulated at the time. The downvotes might be due to extra-mathematical reasons. Happy reading!

The density with respect to the Lebesgue measure $\mathrm dt_1\mathrm dt_2\cdots\mathrm dt_n$ of the distribution of the $n$ first events of the Poisson process is $$ \lambda(t_1)\mathrm e^{-\Lambda(t_1)}\cdot\lambda(t_2)\mathrm e^{-(\Lambda(t_2)-\Lambda(t_1))}\cdots\lambda(t_n)\mathrm e^{-(\Lambda(t_n)-\Lambda(t_{n-1}))}=\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)}, $$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n$, where, for every $t\geqslant0$, $$ \Lambda(t)=\int_0^t\lambda(s)\mathrm ds. $$ Let $T\gt0$ and let $A_n^T$ denote the event that exactly $n$ events of the Poisson process occur in $(0,T]$. Then $A_n^T$ happens if and only if the $n$ first events of the Poisson process happen at some times $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$ and if there is no further event in $(t_n,T]$. Thus, the density of the distribution of the $n$ first events of the Poisson process restricted to the event $A_n^T$ is $$ \lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)}\cdot\mathrm e^{-(\Lambda(T)-\Lambda(t_n))} $$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$, that is, $$ \lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(T)}\cdot\mathbf 1_{0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T}. $$ To sum up, the quantity in the question is the log-likelihood of the $n$ first events of the Poisson process, restricted to $A_n^T$.

One can recover this result by discretization, but not in the limit you suggest. Rather, for every $s\gt0$, consider an independent Bernoulli process $X^s=(X^s_i)_{i\geqslant1}$ such that $p_i^s=\mathbb P(X_i^s=1)$ is $p_i^s=\Lambda(is)-\Lambda((i-1)s)$. Call $(T^s_k)_{k\geqslant1}$ the times when the Bernoulli process $X^s$ is $1$, defined recursively by $T^s_0=0$ and, for every $k\geqslant0$, $$ T^s_{k+1}=\inf\{i\geqslant T^s_k+1\mid X^s_i=1\}. $$ For every $N\geqslant n$, call $A_n^{s,N}$ the event that exactly $n$ Bernoulli random variables in the process $X^s$ up to time $N$ are $1$, thus $A_n^{s,N}=[T^s_n\leqslant N\lt T^s_{n+1}]$. Then the density of $(T^s_k)_{1\leqslant k\leqslant n}$ restricted to the event $A_n^{s,N}$ is $$ \mathbb P((T^s_k)_{1\leqslant k\leqslant n}=(i_k)_{1\leqslant k\leqslant n},A_n^{s,N})=\prod_{k=1}^np^s_{i_k}\cdot\prod_*(1-p^s_i), $$ where $*$ denotes the product over every $1\leqslant i\leqslant N$ except the times $i_k$ for $1\leqslant k\leqslant n$. Call the RHS $R_n^{s,N}(\mathbf i)$ where $\mathbf i=(i_k)_{1\leqslant k\leqslant n}$, then $$ R_n^{s,N}(\mathbf i)=\prod_{k=1}^n\frac{p^s_{i_k}}{1-p^s_{i_k}}\cdot\prod_{i=1}^N(1-p^s_i). $$ Now consider the limit $$ s\to0,\qquad sN\to T\in(0,\infty),\qquad si_k\to t_k,\qquad n\ \text{fixed}. $$ Then, $p^s_{i_k}=\lambda(t_k)s+o(s)$ and $p^s_i=\lambda(is)s+o(s)$ hence $$ \frac{p^s_{i_k}}{1-p^s_{i_k}}\sim\lambda(t_k)s,\quad\prod_{i=1}^N(1-p^s_i)\sim\exp\left(-s\sum_{i=1}^N\lambda(si)\right)\sim\mathrm e^{-\Lambda(T)}. $$ Finally, in the limit considered, $$ R_n^{s,N}(\mathbf i)\sim R_n^T(\mathbf t)\cdot s^n, $$ where $\mathbf t=(t_k)_{1\leqslant k\leqslant n}$, and $$ R_n^T(\mathbf t)=\prod_{k=1}^n\lambda(t_k)\cdot\mathrm e^{-\Lambda(T)}. $$ The log-likelihood in the question is the logarithm of $R_n^T(\mathbf t)$.