Generalized Riemann Integrals

For the first type of improper integrals, $\lim_{x\to\infty}f(x)=0$ is NOT a necessary condition for the convergence of $\int_a^\infty f(x)\, dx$. Consider the characteristic function of the integers. Other kind of examples can be constructed where $f$ is continuous.

For the second type of improper integrals, consider the integral $$\int_0^1 \frac{1}{\sqrt x}\, dx$$ converges but $$\lim_{x\to 0^+} \frac{1}{\sqrt{x}} = \infty$$

Actually, if you require that $$\lim_{x \to b^-} f(x) = 0$$ that means that you could extend $f$ in a continuous way at $b$ as $f(b)=0$ and then $$\int_a^b f(x)\, dx$$ wouldn't even be an improper integral, it would be a regular Riemann integral.