Whether to use "it" or "they" when we talk about using X and Y?

Either version could be right, and the two versions have slightly different meanings. I think the singular version works better.

The subject of the sentence is "using," a singular noun, as further indicated by the fact that you say "is" (generally good practice) not "are" as you would if the subject were plural.

The pronoun (either it or they) is not defined but must refer to some antecedent specified earlier in the sentence.

In the version with "it" the pronoun refers to something singular - that would be "using X and Y", a singular noun.

In the version with "they" the pronoun must refer to something plural - that would be "X and Y".

I think the singular version is better because 1) then the pronoun refers to the subject of the first part of the sentence, which seems more natural and easy to parse, and 2) then the two linked parts of the sentence are more parallel, and both parts are talking about the same thing. So it is a more elegant and less clunky sentence that way.

But, I'd still say, the other version is not grammatically incorrect, and it has a slightly different meaning - that X and Y make things worse, as opposed to using X and Y makes things worse.

This is a great question, because the difference between the two grammatical structures is subtle but conveys distinctly different meaning.

I'll stick with your symbolic examples as they focus on the meaning of the grammar: Using X and Y is generally good practice, but there are also cases where it makes things worse or at least gives nothing. Using X and Y is generally good practice, but there are also cases where they make things worse or at least gives nothing.

In 1., the singular pronoun "it" refers to the singular noun phrase "Using X and Y". When you say "it makes things worse" you mean the whole process of "using-x-and-y" makes things worse.

In 2., the plural pronoun "they" refers to the two discrete nouns within the noun phrase "Using X and Y." In other words, it isn't the whole process that makes things worse, it's using just those specific values for X and Y that makes things worse.

So if So they are both grammatically correct, but because they mean different things, the right choice is the one that says what you really mean.