Can you not sell/destroy buildings?
Solution 1:
There are city projects (all the way at the bottom of the production queue) that allow you to convert the power plant to another power source, e.g. convert to oil, convert to nuclear, ...
These conversions are unlocked through specific research techs.