Proper way to handle plurals with “whose”

The second is correct in this context. Since whose is referring to the plural directories, it must take a plural noun:

What's so special about directories whose names begin with a dot?

The above holds unless the directories collectively have one name, in which case it would be correct to use name in the singular.

Plurality in noun phrases: usually, a plural "possessor" (genitive) goes with a plural possessed noun, even if each possessor only possesses one of the noun in question

The possessive pronoun whose does not have distinct forms for plural and singular, but can represent either. In this case, since it refers back to the plural noun directories, we know that whose must be plural. English does use distinct forms for the personal possessive pronouns its and their, so to answer your question, you can try to use analogy with an independent sentence with the same structure as your relative clause:

1. Their name begins with a dot. (or The name of those directories begins with a dot.)
2. Their names begin with a dot. (or The names of those directories begin with a dot.)

The second seems better to me, and I think to most speakers. The first is definitely grammatical if the directories all have the same name, but that is clearly not the intended meaning in your sentence.

Likewise, we usually use plural nouns like "heads" in sentences like "They covered their heads", even though each person only has one head. (Previous question: "Our head" or "our heads"?)

Plurality in predicates: sometimes a singular can be used distributively with a plural subject, but plural is also an option

There is actually another issue that I haven't seen brought up yet: the use of singular vs. plural in the predicate "begin(s) with a dot". Since the subject "their names" is plural, it is possible to use the plural in the predicate as well: "Their names begin with dots." Unfortunately, because English does not distinguish between collective and distributive plurals, "Their names begin with dots" is ambiguous (it doesn't tell you whether each name begins with a single dot, or with multiple dots). "Their names begin with a dot" is relatively unambiguous (syntactically, it could be understood as meaning that they all collectively begin with the same dot, but there isn't a real chance of confusion in this context), but to some speakers, it may sound a bit strange or awkward because of the mismatch in number. It is sometimes possible to reword to avoid this kind of mismatch, but I don't see an easy way to do that here. Nevertheless, I thought the topic should be mentioned, since your question was about the use of plurals.