What's the difference between "afraid of + verb" and "afraid to + verb"?

With some verbs, there is certainly a distinct difference.

She is afraid of taking the path that leads through the crocodile-infested swamp

would usually be taken as meaning 'She is very worried that she might accidentally choose the dangerous route'.

She is afraid to take the path that leads through the crocodile-infested swamp

means 'She doesn't want to venture that way'.

Afraid to V means anxious, perhaps cripplingly so, about a certain choice. So you wouldn't say 'I'm afraid to slip on a banana skin.'

Afraid of Ving might mean the same, but means 'afraid that N might happen by chance' if that reading is possible. 'Afraid of slipping' means 'afraid that a slip might well occur'. But 'afraid of flying' doesn't have the 'happen by chance' reading available.

As McCawley 1998 puts it (p.126)

Roughly speaking,
• that-complements [tensed clauses] correspond to propositions
• for-to complements [infinitives] correspond to situation types
• 's-ing complements [gerunds] correspond to events

Afraid is a psych predicate adjective (formed from the same root as fear and fright) that takes the experiencer of an emotion as its subject, and a prepositional phrase denoting the source of the fear.

The prepositional phrase must have a noun phrase as its object.

• She's afraid of spiders/that dog/the dark/her shadow.

Gerunds (but not infinitives) may appear as noun phrase objects of prepositions.

• He's afraid of spilling the beans/running a marathon/loving too much/sleeping through it.
• *He's afraid of (to) spill the beans.

Afraid, however, can also take a direct infinitive complement, but without a preposition.

• He's afraid to spill the beans/run a marathon/love too much/sleep through it.

The difference between the gerund and the infinitive constructions with afraid (and most similar predicates) -- to the extent there is any, which isn't much -- is that, using McCawley's "rough" terminology, the infinitive may refer to fear by the subject of the results of the action (a situation type), while the gerund can refer to a fear that the subject may perform the action anyway (an event).

• He's afraid of touching cats, so he wears gloves when they're around.
• He's afraid to touch cats, because he thinks they make you crazy.

But most likely this kind of distinction amounts to very, very fine-scale tuning for most English speakers, and in the right contexts, infinitive and gerund could be turned around with no ill effects.