Is it normal to say "reply distantly" or "reply in a distant voice"?

Solution 1:

A readily-understood trope in English. Here's a good example from The American Civil War of 2018 by B Larson:

        It's been a long time since I've been there,” Ashley replied distantly, as if remembering a long-suppressed memory.
        “You have been around this area a lot, haven't you?” Michael asked, noticing the distant look in her eyes.
        “You could say that...." Ashley said lowly.

Solution 2:

Distant isn't usually used in that manner with the way you've worded it (people would likely interpret it to mean that the reply came from someone who was physically far away, rather than emotionally detached); However, if worded the way Ashley has stated then the meaning you are asking about will be understood.

An alternative might be to say "they replied listlessly" or "they gave a robotic response"