relationship between an increase and decrease?

so I am looking for a word that would best describe such a situation as; the popularity of A decreased because the popularity of B increased.

There are two ways: the second is a restricted sense of the first. It is convenient to illustrate the point with numerical examples.

First: they are certainly inversely related in the loose sense that one goes up as the other goes down.

For example, 48->24->6 (successive decreases of 50% and 75%) in one leads to 24->30->60 (successive increases of 25% and 100%) in the other is simply an inverse relationship.

Second: they may also be inversely proportional if the relative decrease in one is in constant proportion to the relative increase in the other.

For example, 20->10->5 (successive decreases always 50%) in one leads to 7->14->28 (successive increase always 100%) in the other is an inversely proportional relationship. Note that this equates to the reciprocity mentioned by Jim Mack in his answer.

This mathematical use of reciprocal follows is justified from the limited definition:

reciprocal = a number that, when multiplied by another number, results in 1

Cambridge

In this case the product of the two measures of popularity (however it is measured) would be constant (not necessarily 1). Because you give no feeling for what the measure is, and because it seems unlikely to be reliably numeric, this strict mathematical meaning is unlikely (although not impossible) to pertain to your question.