Calculating half derivative of $x$ using Cauchy's repeated integral formula

Admin related note - you shouldn't post a second question if your first question hasn't been answered, you should update the post.

You seem to have discovered that the formula doesn't make sense for $n\le 0$. Here's some things you could do instead.

In the case of $f(x) = x$, you already know that $If = x^2/2, I^2 f = x^3/3!$, and in general $$( I^n f) (x) = \frac{x^{n+1}}{(n+1)!} = \frac{x^{n+1}}{\Gamma(n+2)}$$ This algebraic formula makes perfect sense for $n=-1/2$. So you could set it as the definition of $I^{-1/2}f$: $$ I^{-1/2}f := \frac{x^{n+1}}{\Gamma(n+2)}\Big|_{n=-1/2} = \frac{\sqrt{x}}{\sqrt{\pi}/2}$$
The formula generalises for all $n\in [0,\infty)$ with convergent integrals. So if you want to differentiate halfway, you can differentiate once, then use the formula for $n=+1/2$. This leads to (one choice for) the Caputo fractional derivative of order 1/2: $$ I^{-1/2}f(x):=I^{1/2}(f')(x) = \frac1{\Gamma(1/2)}\int_0^x\frac 1{\sqrt{x-t}}f'(t)dt$$ in the case of $f(x)=x$, $$ I^{-1/2}f(x) = \frac1{\sqrt{\pi}}\int_0^x\frac 1{\sqrt{x-t}}dt =\frac1{\sqrt{\pi}}2\sqrt{x}$$
You could also do it in reverse order - "integrate $\frac12$ times", then differentiate. This is (one choice for) the Riemann-Liouville fractional derivative of order 1/2: $$ I^{-1/2}f(x):= \frac{d}{dx} \frac1{\Gamma(1/2)}\int_0^x\frac 1{\sqrt{x-t}}f(t)dt$$ In the case of $f(x)=x$, $$ I^{-1/2}f(x) = \frac{d}{dx} \frac1{\sqrt{\pi}}\int_0^x\frac 1{\sqrt{x-t}}tdt =\frac1{\sqrt{\pi}} \frac 43 \frac{d}{dx}x^{3/2} = \frac{1}{\sqrt{\pi}} 2\sqrt{x} $$ These formulas happen to match.