It can be used also in a general sense:

lose (all) one's marbles and lose one's mind.

Fig. to go crazy; to go out of one's mind.

  • What a silly thing to say! Have you lost your marbles? Look at Sally jumping up and down and screaming. Is she losing all her marbles? I can't seem to remember anything. I think I'm losing my mind.

(Collins)