Is there an idiom that corresponds to the Hungarian expression "fall off the other side of the horse"?

There's a Hungarian phrase that can be literally translated as something like "fall off the other side of the horse". (The literal implication is either that instead of falling off this side of the horse, you fell off the other side; or that in your zeal to get on the horse, you overshot the target and fell off the other side.) It means that you are at one extreme of a situation, and you want to change it so hard that you fall into the opposite extreme. Is there an equivalent of this phrase/idiom in English?

Simply to 'go from one extreme to the other' or 'go from one extreme to another'.

If that expression means that by going to the other extreme you are still in trouble, this expression might fit:

EDIT:

"out of the frying pan and into the fire"

per @MikeM's comment that I had the expression slightly wrong

The idiom go overboard (“To go to extremes, especially as a result of enthusiasm”) implies extremity, although not necessarily the opposite extremity.

Answering the ‘It means that you are at one extreme of a situation, and you want to change it so hard that you fall into the opposite extreme’ part:

In Old Tales Retold from Grecian Mythology in Talks Around the Fire (1876) by Augusta Larned (as at google books), it is stated that