Spectrum of Indefinite Integral Operators

I'm not sure if this can be done with Fourier series, although that's interesting to think about. Anyway, the usual way to see these things is by applying the spectral theorems for compact operators as is done in what follows.

For $f \in L^2[0,1]$ with $\|f\|_2 \le 1$, the Cauchy-Schwarz Inequality shows that

$$|(Tf)(x) - (Tf)(y)| \le \int_y^x \! |1 \cdot f(t)| \, dt \le \left( \int_y^x \! \, dt \right)^{1/2}\left( \int_y^x \! |f(t)|^2 \, dt \right)^{1/2}$$ $$ \le \|f\|_2|x-y|^{1/2} = |x-y|^{1/2}$$

so $Tf$ is Hölder continuous of exponent $1/2$. It now follows from the Arzela-Ascoli Theorem that $T: L^2[0,1] \to C[0,1]$ is compact and since the identity mapping from $C[0,1]$ to $L^2[0,1]$ is bounded, this shows that $T: L^2[0,1] \to L^2[0,1]$ is compact as well. Thus, by the Spectral Theorem for Compact Operators, all non-zero elements of $\sigma(T)$ are eigenvalues of $T$. Now suppose that $\lambda \neq 0, f \neq 0$ are an eigenvalue and eigenfunction of $T$ respectively. Then

$$\int_0^x \! f(t) \, dt = \lambda f(x)$$

Since the left hand side is continuous, the right hand side must be also so that $f$ is continuous. But since now $f$ is continuous, this makes the left hand side continuously differentiable, so $f$ must be continuously differentiable. We can now use the fundamental theorem of calculus to differentiate both sides thereby getting $f(x) = \lambda f'(x)$ which has only solutions of the form $f(x) = Ce^{t/\lambda}$. But plugging in $x = 0$ on both sides of the equation $Tf = \lambda f$ above, we see that we must also have $f(0) = 0$. Thus $C = 0$ and therefore $f = 0$. This contradiction shows that $T$ has no non-zero eigenvalues and hence we have shown $\sigma(T) = \{0\}$. Now it follows immediately from the definition that the spectral radius $r(T)$ is $0$. This argument does not depend on the fact that $T$ is defined on a Hilbert space, so slight modifications also show that the spectral radius of your second operator is also $0$.

Consider now the function $k:[0,1]^2 \to \mathbb{R}$ defined by $$k(x,y) = \begin{cases} \ 1 & \text{if $x \geq y$},\\ \ 0 & \text{otherwise} \end{cases}$$

Then we have $$Tf = \int_0^x \! f(y) \, dy = \int_0^1 \! k(x,y)f(y) \, dy$$

Thus, it follows by Fubini's Theorem that $$\langle g, Tf \rangle = \int_0^1\! \overline{g(x)}\int_0^1 \! k(x,y)f(y) \, dy \, dx$$ $$ = \int_0^1 \! f(y) \int_0^1 \! \overline{g(x)}k(x,y) \, dx \, dy$$ $$ = \int_0^1\! \overline{\int_0^1 \! g(x) \overline{k(y,x)} \, dx} \, f(y)\, dy$$ $$ = \langle \int_0^1 \! g(x) \overline{k(y,x)} \, dx, f \rangle$$ which shows that we must have

$$(T^*f)(y) = \int_0^1\int_0^1\! \overline{k(y,x)} f(x) \, dx = \int_y^1 \! f(t) \, dt$$

so you can now compute $T^*T$ as

$$T^*Tf(x) = \int_x^1 \int_0^y \! f(t) \, dt dy$$

Finally, to get the norm of $T$ we will use the "$B^*$-Identity" $\|T\| = \sqrt{\|T^*T\|}$. Since $S = T^*T$ is compact and self-adjoint, it's set of eigenvalues has an element of largest absolute value, and this absolute value is equal to the norm of $S$ (Spectral Theorem for Compact Self-Adjoint Operators). Using an argument very similar to the one above showing that $T$ has no non-zero eigenvalues, we see that if $\lambda \neq 0, f \neq 0$ are an eigenvalue and an eigenfunction of $S$ respectively then $f$ is twice continuously differentiable and

$$\lambda f''(x) = - f(x)$$

so that we must have $f(x) = \alpha e^{i \omega x} + \beta e^{-i\omega x}$ for some $\alpha, \beta \in \mathbb{C}$ and $\omega^2 = 1/\lambda$. A direct but somewhat tedious computation yields

$$S(\alpha e^{i \omega x} + \beta e^{-i\omega x}) = \frac{1}{\omega^2}(\alpha e^{i \omega x} + \beta e^{-i\omega x}) + \left(\frac{\alpha}{i\omega} - \frac{\beta}{i\omega} \right)x - \left(\frac{\alpha e^{i \omega}}{\omega^2} - \frac{\beta e^{-i\omega}}{\omega^2}\right) - \left(\frac{\alpha}{i\omega} - \frac{\beta}{i\omega} \right) $$

This formula shows that if $\alpha e^{i \omega x} + \beta e^{-i\omega x}$ is to be an eigenfunction then we must have $\alpha = \beta$ and thus an eigenfunction must be of the form $$f(x) = \alpha(e^{i \omega x} + e^{-i\omega x}) = 2\alpha \cos (\omega x)$$ with eigenvalue $1/\omega^2$. Now we can again just as we did before read from the equation $\omega^2Sf = f$ that $f(1) = 0$, which since $f \neq 0$ (and therefore $\alpha \neq 0$) implies

$$\omega = \frac{(2n + 1)\pi}{2}$$

and thus the eigenvalues are given by

$$\lambda_n = \frac{4}{(2n + 1)^2\pi^2}$$

Maximizing this over $n$ yields a largest eigenvalue of $4/\pi^2$ at $n = 0$. This shows that the norm of $S$ is $4/\pi^2$ and hence the norm of $T$ is $\sqrt{4/\pi^2} = 2/\pi$.

If you want to search for some references about this stuff, the keywords here are "Volterra Operator" and "Hilbert-Schmidt Kernel."

thanks PZZ's answer for the first problem. Now I know why I did realize why I am failed in the second problem.

$$Tf(x)=\int_{0}^{1-x}f(t)dt$$

It is trival that this operator is is linaer compact operator according to Arzela-Ascoli theorem, so we get $\sigma(T)/{0} \subset \sigma_{p}(T)$, use simple computation we can get that $0\notin \sigma_{p}(T)$, for C[0,1] is a infinite dimensional space, we can get $0\in \sigma(T)$. $\forall \lambda \in \sigma_{p}(T)$, and $\lambda \neq 0$, we can get that

$$\int_{0}^{1-x}f(t)dt=\lambda f(x)$$

which implies that $f \in C^{\infty}[0,1]$, then we can get
$$-f(1-x)=\lambda f'(x)$$ notice that $$f(1)=0$$ further we get $$\lambda^{2}f''(x)=-f(x)$$, solve this second order of differential equation, we can calulated every $\lambda_{n}=\frac{2\pi}{2n+1}, \qquad n\in Z$.