Show a convergent series $\sum a_n$, but $\sum a_n^p$ is not convergent

If you allow the $a_n$ to be negative, this is likely to be true for $p$ integers - there's an easy example that diverges for even integers but not for odd integers, and it seems likely that we could find a pleasant example that diverges for all integer $p>2$. (However, as mentioned in the comments, it will be difficult for you to define $a_n^p$ in the case of $p$ not an integer.)

However, if the $a_n$ must be positive, there is no such series. Assume we have such a series; then only finitely many of the $a_n$ are greater than or equal to $1$. Then let's write $b_n$ for the terms of the series that are less than $1$. Then $b_n^p < b_n$, so $$\sum_{n=1}^\infty b_n^p \leq \sum_{n=1}^\infty b_n < \infty$$ by hypothesis (the series is convergent). Since there are only finitely many $a_n \geq 1$, they do not affect the convergence or divergence of the series; so the new series $$\sum_{n=1}^\infty a_n$$ is convergent. This contradicts our hypothesis that it was divergent.

If $p$ is even, make $a_n^p=1/n$, and $a_n$ alternating, that is, let $a_n=(-1)^n/n^{1/p}$.

If $p$ is odd, this is more interesting, because we cannot argue in terms of conditional vs. absolute convergence. This case has been asked before on this site: See here, where I state a very general theorem (with references) and give details for the case $p=3$, that you should be able to adapt easily to cover any odd $p$. The very general result is that for any set $C$ of positive integers there is a sequence $a_0,a_1,a_2,\dots$ of reals such that for any $k$, $$ \sum_n {a_n}^{2k-1} $$ converges iff $k\in C$.