Exceptional case for $\delta$-method

Yes, you should use continuous mapping theorem in this case.

Suppose $\sqrt{n}(\hat{\theta}-\theta)\overset{d}{\to}\mathcal{N}(0,\Sigma)$, and we are interested in $g(\hat{\theta})-g(\theta)$. In the proof of delta method, we first apply a Taylor expansion around $g(\theta)$, which requires $g(\cdot)$ exists at $\theta$.

In a more interesting example (in my opinion), where $\hat{\theta}=(\hat{\theta}_{1},\hat{\theta}_{2})^{\top}$, and $\theta=(0,\theta_{2})^{\top}$,

• If $\theta_{2}\neq0$, delta method gives results like $\sqrt{n}\hat{\theta}_{1}/\hat{\theta}_{2}\overset{d}{\to}\mathcal{N}(0,\sigma^{2})$, hence $\hat{\theta}_{1}/\hat{\theta}_{2}$ is $O_{p}(n^{-1/2})$.
• If $\theta_{2}=0$, continuous mapping theorem gives $\hat{\theta}_{1}/\hat{\theta}_{2}\overset{d}{\to}\mathrm{Cauchy}$, hence $\hat{\theta}_{1}/\hat{\theta}_{2}$ is $O_{p}(1)$.