Limit using Itô isometry
Solution 1:
Using the hint by @SangchulLee we can prove the required limit:
$$\begin{align*} E\left[\left(\int_0^t\frac{B_s}{\epsilon}\chi_{B_s\in(-\epsilon,\epsilon)}dB_s\right)^2\right] & = E\left[\int_0^t \left(\frac{B_s}{\epsilon}\right)^2\chi_{B_s\in(-\epsilon,\epsilon)}ds\right] & \text{(Itô isometry)} \\ &= \int_0^t \epsilon^{-2} E\left[B_s^2\chi_{B_s\in(-\epsilon,\epsilon) } \right] ds & \text{(Tonelli's theorem)} \\ &\leq \int_0^t \epsilon^{-2} \int_{-\epsilon}^\epsilon \frac{x^2}{\sqrt{2 \pi s}}dx ds \\ &= \int_0^t \epsilon^{-2} \frac{2}{3\sqrt{2 \pi s}} \epsilon^3 ds \\ &= \epsilon \cdot C \sqrt{t} & \text{(for some constant }C)\\ &\to 0 & \text{(as } \epsilon \to 0^+ ) \end{align*}$$
This implies that $\int_0^t\frac{B_s}{\epsilon}\chi_{B_s\in(-\epsilon,\epsilon)}dB_s \to 0$ in $L^2(P)$ as $\epsilon \to 0^+$, as desired.