Does $f'(x)\in \mathbb Z$ a.e. implies that $f$ is an affine function?

Yes, $f'$ must be constant. This has been answered in the comments; the result in https://mathoverflow.net/questions/266377/how-quickly-can-the-derivative-of-an-everywhere-differentiable-function-change-s gives a kind of positive measure Darboux theorem. This is a special case of How irregular can $f'$ be beyond Darboux's Theorem?