rains hard enough to cause all the damage

though I couldn't remember rains hard enough to cause all the damage.

"Rains" here is not a verb but rather a noun. A "rain" is a noticeable event of precipitation. So the writer could not remember events of rainfall that were severe enough to wash out places in the road (which was presumably constructed of gravel or dirt).

I've seen some rains here in southern Minnesota this summer which severely rutted gravel driveways (and probably some rural gravel roads).