Closure of sets A and B

I assume you know that $\overline{A}$ is the smallest (regarding set inclusion) closed set containing $A$.

Fix $x\in(a,b)$. For every $\epsilon>0$, $$\lvert f'(x)-\frac{f(x)-f(y)}{x-y}\rvert<\epsilon$$ provided only that $y\not=x$ is sufficiently close to $x$, so that $f'(x)\in\overline{B}$. This means that $A\subset\overline{B}$ and hence $\overline{A}\subset\overline{B}$.

Conversely, let $x,y\in(a,b)$, $x\not=y$. By the mean value theorem, there exists $\xi$ between $x$ and $y$ such that $$\frac{f(x)-f(y)}{x-y}=f'(\xi)\in A\mbox{.}$$ Thus $B\subset A$ and hence $\overline{B}\subset\overline{A}$.