Is it ever explained why Gorons are allowed in Gerudo Town?

There are two gorons inside Gerudo town, who were allowed inside Gerudo Town, despite being men. One of them even wonders himself why he was allowed entry in the first place.

Is it ever explained somewhere why they were allowed in? Like... are Gorons and Gerudo best buddies or something?

I believe another Gerudo in the town says that they are not sure whether the Gorons they let in are male or female. So, it seems to simply be that the Gerudo cannot tell the difference between male and female Gorons and err on the side of caution by letting them in.

In the japanese version of the game, after you free Nabooris, a second Goron can be seen alongside Lyndae. Talk to them, and they will say "I made a new friend! But are they a boy friend or a girl friend?"

I stole the image above from this Tumblr post, which elaborates further on that.

So this was dropped from the English localization. It seems that Gorons are genderless. You may think of them as males because of their low voice and bulky, muscular bodies, but that is just appearances and first impressions.

Gerudo people apparently will accept genderless people among them since their rule is no males allowed. Having no gender is different from being male.