What is the rule for pronouncing the plural of short words ending with -th?

I've found out that with some words, the /θ/ sound becomes voiced when the -s suffix is added:

• path -> /pæðz/
• bath -> /bæðz/
• youth -> /juðz/
• oath -> /oʊðz/
• mouth -> /maʊðz/
• truth -> /truðz/

and with others, it doesn't:

• myth -> /mɪθs/
• breath -> /breθs/
• month -> /mʌnθs/

Is there any rule behind it, or is it by pure chance?

For native speakers: If you came across a word like that that you're not familiar with, would you be able to correctly guess the pronunciation of its plural?

There isn't any particularly simple rule that accounts for all cases. One rule that Wikipedia gives that I have not been able to find a counterexample for is that /θs/ is always used after a consonant.

The Wikipedia article "Pronunciation of English ⟨th⟩" has a (fairly under-sourced) section about plurals ending in ⟨ths⟩. It says

Plural ⟨s⟩ after ⟨th⟩ may be realised as either /ðz/ or /θs/:

• Some plural nouns ending in ⟨ths⟩, with a preceding vowel, have /ðz/, although the singulars always have /θ/; however a variant in /θs/ will be found for many of these: baths, mouths, oaths, paths, sheaths, truths, wreaths, youths exist in both varieties; clothes always has /ðz/ (if not pronounced /kloʊz/, the traditional pronunciation).

• Others have only /θs/: azimuths, breaths, cloths, deaths, faiths, Goths, growths, mammoths, moths, myths, smiths, sloths, zeniths, etc. This includes all words in 'th' preceded by a consonant (earths, hearths, lengths, months, widths, etc.) and all numeric words, whether preceded by vowel or consonant (fourths, fifths, sixths, sevenths, eighths /eɪtθs/, twelfths, fifteenths, twentieths, hundredths /hʌndrədθs/, thousandths).

• Booth has /ð/ in the singular and hence /ðz/ in the plural for most speakers in England. In American English, it has /θ/ in the singular and /θs/ or /ðz/ in the plural. This pronunciation also prevails in Scotland.

Another section of the article says

In Scottish English, /θ/ is found in many words which have /ð/ further south. The phenomenon of nouns terminating in /θ/ taking plurals in /ðz/ does not occur in the north. Thus the following have /θs/: baths, mouths (noun), truths. Scottish English does have the termination /ðz/ in verb forms, however, such as bathes, mouths (verb), loathes, and also in the noun clothes

The Oxford Reference Guide to English Morphology (2013), by Laurie Bauer, Rochelle Lieber, and Ingo Plag, says

Where /θ/ is voiced to /ð/ in the formation of the plural, there is no reflection of the difference in spelling. Accordingly, there is little normative pressure on these words. [As with words ending in /f/,] we find words which have the voiced plural, words which fluctuate between the voiced and the regular plural, and words which show the regular /θ/ plural.

(24)
a. plurals in /ðz/: booth (for those who pronounce it with final /θ/), mouth, youth ('young man')
b. both plurals found: lath, oath, path, sheath, truth, wreath
c. plurals in /θs/: berth, birth, breath, cloth, death, depth, earth, faith, growth, heath, length, month, moth, smith, strength, tenth (and all similar fractions), youth ('young days'), -path, hyacinth, myth, shibboleth, zenith, and foreign words in -lith, -path etc.

The word bath has in BrEng the plural /ba:ðz/ when it refers to a public swimming pool, but /ba:θs/ when it refers to domestic installations.

[...]

Many of the /θs/ and /ðz/ clusters are variably simplified in all but the most formal styles with the omission of the dental fricatives (and perhaps some compensatory lengthening of the alveolar fricative)

(p. 130-131)

The authors give the example clothes to illustrate this last point, although I don't think that's a great example of synchronic simplification of /θs/ or /ðz/ to /s/ or /z/. A better example in my opinion is the commonly noted pronunciation of months, the plural of month /mʌn(t)θ/, as something like /mʌn(t)s/.

How those plurals are pronounced is more likely to differ by regional accents as there is no explicit rule for saying them one way or the other. The distinction between those sounds is unimportant in English.