People who do not drink tea
The short answer to your question is no, there is no specific term for people who do not drink tea or coffee. The general term for someone who avoids something is abstainer.
abstainer, noun.
- a person who abstains from something regarded as improper or unhealthy, especially the drinking of alcoholic beverages.
- a person who abstains from anything.
abstain, verb (used without object).
- to hold oneself back voluntarily, especially from something regarded as improper or unhealthy (usually followed by from)
There is a word you can use, but it is not an established word:
teatotaller
This is based on the same-sounding word
teetotaller
which is someone who abstains from alcohol (link to Oxford dictionary online).