Idiom for expressing feeling of not willing to talk to a person who always replies in a disrespectful manner?

Is there an idiom for expressing the feeling of not willing to talk to a person who always replies in disdain or have such attitude or replies back in a disrespectful manner?

My first thought was "tread lightly."

From Cambridge Dictionaries online "tread carefully/gently/lightly":

to speak or behave carefully to avoid upsetting or causing offence to anyone

The Free Dictionary defines "tread carefully also tread warily" as follows:

to avoid saying or doing anything that could cause difficulties

It gives the following usage note, which comes close to fitting your purposes:

sometimes used in the form tread cautiously: You should tread cautiously when discussing financial matters with him.