Matrix given by $a_{ij} = 1/(i+j)$ is non-singular.
Solution 1:
About determinant of $A$, it is a special case of something called Cauchy's double alternant.
In general, given any $2n$ numbers $x_1, \ldots, x_n, y_1, \ldots, y_n$. If one construct a $n \times n$ matrix with entry $\frac{1}{x_i + y_j}$ at $i^{th}$ row, $j^{th}$ column, one has $$\det\left[ \frac{1}{x_i + y_j}\right] = \frac{\prod_{1\le i < j\le n}(x_i - x_j)(y_i - y_j)}{\prod_{1\le i, j \le n}(x_i + y_j)}\tag{*1}$$ In particular, this means if $x_i$ and $-y_j$ are $2n$ distinct numbers, the determinant is non-zero and hence the matrix is invertible.
It is not that hard to prove $(*1)$ ourselves, consider the expression
$$\prod_{1\le i, j \le n}(x_i + y_j)\times\det\left[ \frac{1}{x_i + y_j}\right]$$
If one expand the determinant, it is easy to see this expression is a polynomial in $x_i, y_j$. Since the determinant vanishes whenever $x_i$ equals to some $x_j$ or $y_i$ equals to some $y_j$. This polynomial contains $\prod_{1\le i < j\le n}(x_i - x_j)(y_i - y_j)$ as a factor. By matching the degree of polynomial on both sides, we find
$$\det\left[ \frac{1}{x_i + y_j}\right] = \lambda_n \frac{\prod_{1\le i < j\le n}(x_i - x_j)(y_i - y_j)}{\prod_{1\le i, j \le n}(x_i + y_j)}$$
for some constant $\lambda_n$ depends only on $n$. If one replace $x_1$ by $\epsilon x_1$, $y_1$ by $\epsilon y_1$, send $\epsilon$ to $0$ and look at the limiting behaviors of both sides, one can conclude $\lambda_n = \lambda_{n-1}$. One can verify $\lambda_1 = \lambda_2 = 1$ by hand and hence $\lambda_n = 1$ for all $n$.
Apply this to the Hilbert like matrix $A$ at hand, we get
$$\det\left[\frac{1}{i+j}\right] = \frac{(c_n c_{n+1})^2}{c_{2n+1}} \ne 0 \quad\text{ where }\quad c_n = \prod_{k=1}^{n-1} k! $$ and hence $A$ is invertible.
Solution 2:
Here is one approach:
You know that $x \mapsto x^k$, $k=1,2,...$ are linearly independent and so the operator $L: \mathbb{R}^n \to \mathbb{R}[x]$ given by $(L \alpha) (x) = \sum_k \alpha_k x^k$ is injective. Hence $L^*L$ is injective, and since $A= L^*L$ we see that $A$ is invertible. (The inner product specified in the question is used implicitly throughout this answer.)
Here is an alternative approach to the underlying problem:
Let $s_n(x) = 1-(1-x)^n$. Note that $\|s_n\| \le 1$, and $s_n(x) \to 1_{(0,1]}(x)$ for all $x \in [0,1]$ (and convergence is uniform on compact subsets of $(0,1]$).
In particular, we have $s_n \cdot p \in U$ for all $p \in \mathbb{R}[x]$, and $\|p-s_n \cdot p \| \to 0$. It follows that $\overline{U} = \mathbb{R}[x]$.
Since $U^\bot = \overline{U}^\bot$, we see that $U^\bot = \{ 0 \} $.
As a related aside, note that the 'evaluation' operator $Ep = p(0)$ is not continuous with the given norm.