Representation of $e$ as a descending series

You can derive the following telescoping sum for $k\ge2$:

\begin{align*} \frac{1}{k!}+\frac{1}{k!(k-1)k} &=\frac{1}{k!}\left(1+\frac{1}{(k-1)k}\right)\\ &=\frac{1}{k!}\left(1+\frac{1}{k-1}-\frac{1}{k}\right)\\ &=\frac{k-1}{kk!}+\frac{1}{(k-1)k!}\\ &=\frac{k-1}{kk!}+\frac{1}{(k-1)k(k-1)!}\\ &=\frac{k-1}{kk!}+\frac{1}{(k-1)(k-1)!}-\frac{1}{k(k-1)!}\\ &=\frac{1}{k!}-\frac{1}{kk!}+\frac{1}{(k-1)(k-1)!}-\frac{1}{k!}\\ &=\frac{1}{(k-1)(k-1)!}-\frac{1}{kk!} \end{align*}

Hence, if you go back to the original series representation of $e$ as you did:

\begin{align*} e+\sum_{k=2}^{\infty}\frac{1}{k!k(k-1)} &=2+\sum_{k=2}^{\infty}\frac{1}{k!}+\frac{1}{k!k(k-1)}\\ &=2+\sum_{k=2}^{\infty}\frac{1}{(k-1)(k-1)!}-\frac{1}{kk!}\\ &=2+1-\lim_{n\rightarrow\infty}\frac{1}{nn!}=3 \end{align*}

As an alternative approach, we have $$ 3-e=\int_{0}^{1}x(1-x)e^x\,dx \tag{1}$$ by IBP and the RHS of $(1)$ is clearly positive, hence $e<3$.
Since $\int_{0}^{1}x(1-x)\frac{x^k}{k!}\,dx = \frac{1}{k!(k+2)(k+3)}$ we also have $$ 3-e = \sum_{k\geq 0}\frac{1}{k!(k+2)(k+3)}\tag{2} $$ by termwise integration of a Taylor series.

An equivalent integral

Plugging a relationship similar to the one used by Jack D'Aurizio, namely $$\frac{1}{k!(k-1)k}=\int_0^1 \frac{x^{k-2}(1-x)}{k!} dx$$ into this integral

$$\int_0^1 \frac{(1-x)(e^x-1-x)}{x^2}dx = 3-e$$

with non-negative integrand in $(0,1)$ that proves $e<3$, yields the series in the question.

$$\sum_{k=2}^\infty \frac{1}{k!(k-1)k}=3-e$$

This relates the series with the inequality $1+x \leq e^x$.

Similar approximations

This integral is similar to the one used to explain why $e$ is close to the eighth harmonic number.

$$\frac{1}{14} \int_0^1 x^2(1-x)^2(e^x-1-x)dx = e-\frac{761}{280}=e-H_8\approx 0$$

with corresponding series $$e=\frac{761}{280}+\frac{1}{7}\sum_{k=2}^\infty \frac{1}{k!(k+3)(k+4)(k+5)}$$

and $$\frac{1}{2}\int_0^1 (1-x)^2\left(e^x-1-x-\frac{x^2}{2}\right)dx = e-\frac{163}{60}$$

used to explain the observation by Lucian that $2\pi+e$ is close to $9$.

Another series for $3-e$

Yet another series to prove $e<3$ is related to the integer sequence http://oeis.org/A165457.

$$\frac{1}{e}=\frac{1}{3}+\sum_{k=1}^\infty \frac{1}{(2k+1)!(2k+3)}$$