Emergence of Cauchy Principal value

Yes, your observation is precisely that Fubini does not apply, because one of the iterated integrals is (sometimes) infinity. But you can cut off a small ball around $\newcommand{\o}{\omega_{10}}\o>0$ before interchanging the integrals: \begin{align} \int_0^t \int_0^\infty v(\omega)e^{i(\o-\omega)t}d\omega dt &= \lim_{\epsilon\to 0}\int_0^t\int_{\omega\in[0,\infty),|\omega-\o|>\epsilon} v(\omega)e^{i(\o-\omega)t}d\omega dt \\ &= \lim_{\epsilon\to 0}\int_{\omega\in[0,\infty),|\omega-\o|>\epsilon} \int_0^tv(\omega)e^{i(\o-\omega)t}d\omega dt \\ &= \lim_{\epsilon\to 0}\int_{\omega\in[0,\infty),|\omega-\o|>\epsilon} \frac{v(\omega)e^{i(\o-\omega)t}}{i(\o-\omega)}d\omega \\&=:\mathcal P\!\!\!\int_0^\infty \frac{v(\omega)e^{i(\o-\omega)t}}{i(\o-\omega)}d\omega \end{align} The above works only when $\o\neq 0$. Also, if $v(\o)=0$ and is $C^1$ then the above still works but then the integral does exist, so $\mathcal P\!\!\int_0^\infty=\int_0^\infty$.