What is a trans step-parent called?

The term stepmother is appropriate here. After all, it was Caitlyn's decision to transition to a woman*, and she appears to have adopted all the feminine terminology (for example, the pronoun "she"). Or, to be safe, you can refer to her as a stepparent. You could also just flip the entire thing and mention that Kim is Caitlyn's stepdaughter (as most tabloids seem to do).

I did find an article that uses "stepmother":

Reality TV star Kim Kardashian says that she first laid eyes on her stepmother Caitlyn, who was previously her father Bruce Jenner before undergoing a gender transition, when she attended the cover shoot of the magazine and she loved it.

_{Caitlyn Jenner is 'Beautiful', Says Step-Daughter Kim Kardashian}

It's not unheard of to have two mothers (or fathers), with gay couples being the more prominent example.

In either situation, however, society tends to jump to cis, heteronormal conclusions. This fact is illustrated by the experiences of a man with an ex-wife, who has married another man named Dean:

Indeed, there have been occasions in which Dean has been introduced as a “stepparent” by my children in public settings when their mother and I were around. The common assumption? That Dean was now married to their mom rather than in relationship with me, because stepparents are an accepted norm in a straight world, and still an oddity in the world of same-sex parents. This would both bring gales of laughter along with the sad recognition that we live in a world in which only children with straight couples who remarry can have stepparents, leaving the LGBTQ stepparents out in the dust.

_{The Complicated Role of a Same-Sex Stepparent}

Similar ambiguity exists even outside of LGBTQ contexts with aunts and uncles: you don't know who's a spouse and who's a sibling. The exact relationship can only be expressed in multiple words, though it often isn’t necessary to do so.

* Caitlyn fits into the most common category of transgenderism: her biological gender at birth was "wrong" and she was really the opposite gender. Of course, gender isn't always binary... Or constant, for that matter either (genderfluid).

In general, you should respect a person's preference for pronouns and other gender-specific language. Ideally, you would ask them (if they haven't already told you... or Vanity Fair, in Caitlyn's case) what they prefer. There are a lot of different terms that may be used: here's a list of some.

Caitlyn would probably still go as stepmother no matter what, since her entire gender basically changed. In case you're confused with how to treat someone as a transgender, it's basically just treating them as the gender they identify as. So I would likely just call Caitlyn the stepmother of Kim and her siblings.

One may choose to use the context of time to decide. If the frame of reference was before the operation, one would say stepfather. If the frame of reference was after the operation, and he prefers to be called stepmother, and the communicator chooses to disregard genetics and override with Bruce/Caitlyn's preference, then one would use stepmother.