Deriving an exact confidence interval for parameter of an exponential random variable

Yes, your work is correct for an exact, equal-tailed confidence interval; that is, the probability that the interval exceeds the true parameter equals $\alpha/2$, which is also the probability the interval is below the true parameter. One could construct an interval that does not allocate the total error $\alpha$ equally, but still has the same coverage probability $1 - \alpha$.

An alternate formulation of the lower and upper confidence limit can be obtained by noting that the distribution of $1/\bar X$ is inverse gamma with PDF $$f_{\theta_{\text{MLE}}} (s) = \frac{(n \theta/s)^n e^{-n \theta/s}}{s \Gamma(n)}, \quad s > 0.$$ Thus the lower and upper limits may be expressed directly in terms of quantiles of this distribution, namely $$\Pr\left[F_{\theta_{\text{MLE}}}^{-1}(\alpha/2) \le \theta \le F_{\theta_{\text{MLE}}}^{-1}(1 - \alpha/2)\right] = 1 - \alpha.$$