What is a Gibbering Gemstone used for?
It is a component required to create the item "Staff of Herding."
Other Items required are: Black Mushroom, Wirt's Bell, Liquid Rainbow, Leoric's Shinebone, the crafting plans, and 50,000 gold.
The gold value required goes up per difficulty and the plans required are "Nightmare Staff of Herding" and etc.
This is one of the items needed to access the Secret Cow Level. Judging from your question, you haven't found anything else related to the quest yet, so the most you can do is hold onto it for the time being.