Are the rhetorical meanings of "elliptic", "hyperbolic", and "circular" connected to their mathematical meanings?

Yes, all the mathematical meanings are related to their respective rhetorical meanings, as seen in etymonline entries for stem parts parabol, ellipse, hyperbol, circul, and circle.

It appears that the mathematical meanings follow the non-mathematical ones, in the sense that the conic section names are based on metaphorical usage of pre-existing terms from the Greek. For example, in the ellipse entry mentioned above, we see:

Ellipse, 1753, from Fr. ellipse (17c.), from L. ellipsis "ellipse," also, "a falling short, deficit," from Gk. elleipsis (see ellipsis). So called because the conic section of the cutting plane makes a smaller angle with the base than does the side of the cone, hence, a "falling short." First applied by Apollonius of Perga (3c. B.C.E.).

and also

Ellipsis, 1560s, "an ellipse," from L. ellipsis, from Gk. elleipsis "a falling short, defect, ellipse," from elleipein "to fall short, leave out," from en- "in" + leipein "to leave" (see relinquish). Grammatical sense first recorded 1610s.