Lose one's marbles
It can be used also in a general sense:
lose (all) one's marbles and lose one's mind.
Fig. to go crazy; to go out of one's mind.
- What a silly thing to say! Have you lost your marbles? Look at Sally jumping up and down and screaming. Is she losing all her marbles? I can't seem to remember anything. I think I'm losing my mind.
(Collins)