Why is $\zeta(s)=\lim_{x\to\infty}\left(\sum_{n\leqslant x} \frac{1}{n^s}-\frac{x^{1-s}}{1-s}\right)$ for $0<s<1$?

I am currently reading the book Introduction to Algebraic Number Theory by Apostol. To introduce some important asymptotic formulas, Apostol gives a rough definition of the Riemann zeta function (for $s\in\mathbb{R}^+$),

$$\begin{equation}\zeta(s)=\begin{cases} \sum_{n}n^{-s}, &s>1\\ \lim_{x\to\infty}\left(\sum_{n\leqslant x} \frac{1}{n^s}-\frac{x^{1-s}}{1-s}\right), &0<s<1 \end{cases}\end{equation}$$

The second part really confused me. How could we approach this limit? If we see $\zeta$ as an analytic continuation of $\sum_{n}\frac{1}{n^s}$, it should be written as $$\zeta(s)=\frac{1}{\Gamma(s)}\int_0^\infty \frac{x^{s-1}}{e^x-1} dx$$ This formula can be easily derived from $\Gamma(s)=\int_0^\infty x^{s-1}e^{-x}dx$ by substituting $x=nu$ (which was exactly what Riemann did in his paper). However, I don't see the connection between this formula and the limit form for $0<s<1$. I am really new to this function so maybe this is a dumb question. But please point it out why we can write $\zeta(s)$ in the limit form for real $0<s<1$.

Also, historically, is the limit form derived from the formal or the converse?

Thanks in advance, any help will be appreciated.

For an elementary approach, you want to show that the limit is indeed analytic in $s$ (uniform limit of analytic functions) in an open subset of $D=\{s:\Re(s)>0\land s\ne1\}$. For $\Re(s)>1$ this is fairly trivial since $x^{1-s}\to0$. For $0<\Re(s)\le1$, a full asymptotic expansion makes this more obvious, but it suffices to simply bound the error between the given sum and

$$\int_0^x\frac{\mathrm dt}{t^s}=\frac{x^{1-s}}{1-s}\tag{$0<\Re(s)\le1,s\ne1$}$$

using something such as Taylor expansions.

A much more general approach is given by the Euler-Maclaurin summation formula, which states that

$$\sum_{n\le x}\frac1{n^s}=\zeta(s)+\frac1{(1-s)x^{s-1}}+\frac1{2x^s}-\frac s{12x^{s+1}}+\mathcal O(x^{-s-3})$$

For $\Re(s)>1$, every term after $\zeta(s)$ tends to zero, so we get

$$\lim_{x\to\infty}\sum_{n\le x}\frac1{n^s}=\zeta(s)$$

For $\Re(s)>0$, the $x^{-s+1}$ term needn't go to zero, so we get

$$\lim_{x\to\infty}\left[\sum_{n\le x}\frac1{n^s}-\frac1{(1-s)x^{s-1}}\right]=\zeta(s)$$

In general, by moving all terms which don't go to zero to the other side, we may get a converging limit expression for $\zeta(s)$ for $\Re(s)>-N$ for any natural $N$. It is interesting to note that this gives exacts when $s$ is a negative integer since $\sum_{n\le x}n^{-s}$ has a closed form.