Is there a better alternative to the phrase, 'it holds that'?
Solution 1:
Personally, I would recommend not saying things like "It holds that" which few, if any, native English speakers would say. Instead, say "we have that" or just "we have", or perhaps replace "It holds that x > y" with "The inequality x > y holds".
Similarly, I would recommend not saying things like "We denote by x the distance between y and z". Instead, say "We let x denote the distance between y and z".
Solution 2:
I don't see anything wrong with such phrases. I think they're perfectly idiomatic, and I don't see them as long-winded. I'd much rather occupy another couple centimeters of space on the page than confuse the reader with an ambiguous statement.
In the second case, I personally prefer "we have that" or "we have". I don't care as much for "it holds that", because my brain briefly searches for the antecedent of "it" and there's a moment of grammatical dissonance.
Solution 3:
Technically speaking, whatever the property/predicate/implication following
"for all [(such and such) $\in$ (domain)], ..."
...logically speaking, it needs NO connecting phrase or words; a comma will often do.
But admittedly, when writing mathematical exposition, I think it helps with parsing to use an "intermediate" phrase: "for all [such and such], we have that...". Whether, or what to use, depends on the context, the number of quantified variables, and the complexity of the predicates which follow. But my own preference is ", ...we have that"... And although it does strike me as "repetitive" in my own writing (in that I use it so often), I wouldn't worry about it being "long-winded."
- I also often write "$\exists x$ such that..." or "$\exists x$ for which...", even though it may not, technically, be necessary.