Why are present participle and infinitive equally acceptable for some verbs, but not others?

This question about "started teaching/to teach" made me realise that even though the present participle and infinitive are both acceptable after "started", that's not the case with other superficially equivalent "auxiliary" verbs...

She began crying/to cry. (both okay - and to me, mean exactly the same thing)

She ceased caring/to care. (both okay, but the infinitive seems slightly more "natural" to me)

It commenced raining/?to rain.

It stopped raining/*to rain.

He quit gambling/*to gamble.

We gave up trying/*to try.

What is it about those last four that prevents the infinitive form being acceptable?

I have a vague sense that in some usages the infinitive might have been more acceptable in the past ("It commenced to rain" sounds a bit "Victorian" to me), but I can't square that with my preference for "I ceased to care" over "I ceased caring".

Solution 1:

Some verbs may be followed by a gerund if they are describing an "actual, vivid or fulfilled action" (Frodesen); "cease" is one of this verbs. Probably your preference stems from this.

Also, 'begin' and 'commence' may be followed by a gerund if they are describing an "actual, vivid or fulfilled action" (Frodesen).

The following verbs can be followed by either an infinitive or a gerund, but there will be a difference in meaning. I stopped smoking means something quite different, for instance, from I stopped to smoke. The infinitive form will usually describe a potential action.

• forget
• remember
• stop

Solution 2:

Different verbs allow different types of complements. Some take objects, and some don't. Some take that content clauses, some take whether content clauses, and some don't take any content clauses. Some take infinitives, and some take gerunds. That's just the way language is.

Solution 3:

This will only partly answer an otherwise loaded question.

Some phrases acquire an idiomatic sense and when we hear them, we instinctively expect a certain related thing to follow.
I stopped to ...: To..., do what?
Rather than a quirk of the language, it is because of the way the brain works: by fuzzy, non-linear, logic.

Trying to find a rational explanation in English grammar would, therefore, be futile.