How prove this $(p-1)!\left(1+\frac{1}{2}+\frac{1}{3}+\cdots+\frac{1}{p-1}\right)\equiv 0\pmod{p^2}$

The solution below is adapted from Notes on Wolstenholme’s Theorem by Timothy H. Choi.

Let $$ S=(p-1)!\sum_{k=1}^{p-1} \frac1k $$ Using your insight $$ \dfrac{1}{k}+\dfrac{1}{p-k}=\dfrac{p}{k(p-k)} $$ we have $$ 2S=(p-1)!\sum_{k=1}^{p-1} \left(\dfrac{1}{k}+\dfrac{1}{p-k}\right) = p\sum_{k=1}^{p-1} \frac{(p-1)!}{k(p-k)} = pS' $$ Note that $S'$ is an integer. Now $$ \frac{(p-1)!}{k(p-k)} \equiv (k^2)^{-1} \bmod p $$ where the inverse is taken ${}\bmod p$. This is a consequence of Wilson’s Theorem. Hence $$ S'\equiv \sum_{k=1}^{p-1} (k^2)^{-1} \equiv \sum_{k=1}^{p-1} k^2 = \frac{(p-1)p(2(p-1)+1)}{6} \equiv 0 \bmod p $$ This means that $2S\equiv 0 \bmod p^2$ and so $S\equiv 0 \bmod p^2$. (We need $p>3$ twice here.)

As others have noted, the congruence is not true for $p=3$, since $$ 2!\left(1+\frac 1 2\right)=2+1=3,$$ which is not divisible by $9$. We can still use what you suggested to prove the congruence holds $\operatorname{mod} p$. Let $p$ be an odd prime, then \begin{align*} (p-1)!\sum_{k=1}^{p-1} \frac 1 k &= (p-1)! \sum_{k=1}^{(p-1)/2} \left(\frac 1 k + \frac 1 {p-k}\right) \\&= (p-1)!\sum_{k=1}^{(p-1)/2} \frac{p}{k(p-k)} = p\sum_{k=1}^{(p-1)/2} \frac{(p-1)!}{k(p-k)}. \end{align*} Since $(p-1)!$ always contains $k$ and $(p-k)$ as a factor, the fractions in the sum are integers and the result is a multiple of $p$.

See lhf's answer for why it is even a multiple of $p^2$ as long as $p>3$.

I came up with this proof while I work on this problem.

We write $f(x) \equiv_p g(x)$ if polynomials $f(x), g(x) \in \mathbb{Z}[x]$ satisfies $a_i\equiv b_i$ mod $p$ for all coefficients $a_i$ of $f(x)$, and $b_i$ of $g(x)$.

Then we have by Fermat's theorem, $$ x^{p-1}-1 \equiv_p (x-1)(x-2) \cdots (x-(p-1)). $$

Group the numbers in pairs as discussed here already, $$ \frac 11+\frac 1{p-1}=\frac p{1(p-1)}, \ \ \frac12+ \frac 1{p-2}=\frac p{2(p-2)}, \ \cdots, \ \frac 1{(p-1)/2}+\frac1{(p+1)/2}=\frac p{ (p-1)(p+1)/4 }. $$ Each group has numerator divisible by $p$. Then it suffices to prove that the numerator $N$ in $$ \sum_{j=1}^{(p-1)/2} \frac 1{j(p-j)}=\frac N{(p-1)!}\ \ \ (*) $$ is divisible by $p$.

We may write

\begin{align*}x^{p-1}-1&\equiv_p (x-1)(x-2)\cdots (x-(p-1))\\ &= \prod_{j=1}^{(p-1)/2} (x-j)(x-(p-j)) \\ &=\prod_{j=1}^{(p-1)/2} (x^2-(j+(p-j))x + j(p-j))\\ &\equiv_p\prod_{j=1}^{(p-1)/2} (x^2+j(p-j)) \end{align*} Expanding the last product, we find that $N$ is the coefficient of $x^2$. Since $p\geq 5$, the coefficient of $x^2$ in the product must be divisible by $p$. Hence, $p|N$ as desired.