Comma usage with compound sentences {actually compound predicates as per Rob's answer}
Solution 1:
The original sentence has a compound predicate. A compound sentence, by contrast, is two sentences joined by punctuation.
Most styles wouldn't use a comma to separate the parts of a compound predicate (or, put another way, to separate the subject of a sentence from any of its verbs). I would remove the comma, or else I'd keep it and then repeat the subject (or a pronoun of it): "They usually start out as an abbreviation, but they turn into acronyms when they become well known."
See, for example, 6.29 in the Chicago Manual of Style (16th edition): "A comma is not normally used between the parts of a compound predicate—that is, two or more verbs having the same subject, as distinct from two independent clauses (see 6.28)."
Solution 2:
I think your problem is with the logic. What the compound sentences rule you're talking about is really saying is:
Rule 1: If a sentence is a compound sentence, then the conjunction should be accompanied by a comma (most of the time).
But the sentence you gave as an example isn't a compound sentence, i.e. it doesn't have two independent clauses, so the compound sentences rule doesn't apply.
I think the mistaken logical thinking you've been doing is to say, "Since this isn't a compound sentence, a comma must not be used." In other words, you've misinterpreted Rule 1 to mean:
Rule 2: If the conjunction is accompanied by a comma, then the sentence is a compound sentence.
You feel bothered, because you know your example sentence isn't a compound sentence!
However, Rule 2 does not follow logically from Rule 1.
Let's see what's going on in your example sentence.
They usually start out as an abbreviation, but turn into acronyms when they become well known.
Here we have a shared subject for the two main verbs. This will be clearer if I simplify the sentence and replace but with and:
They start out as A and turn into B.
The comma is optional here. I like it in your example sentence because the sentence is on the long side, and the comma gives us a nice small rest. Also, the comma accentuates the contrast.