How to compactly write a simple set for 2 variables when second variable may have same properties as first or it may optionally be positive infinity?

Neither of your sets is written correctly. Mathematical notation is not a shorthand for English text and therefore, different rules apply:

1. “$n, h \in \mathbb Z \geq 0$” does not make sense. When a statement contains several relation symbols (like $\in$ and $\geq$ in this case), the normal way to interpret this is as two separate statements like this: $n, h \in \mathbb Z$ (which makes sense) and $\mathbb Z \geq 0$ (which does not). [Technically, even something like $a < b < c$ is abuse of notation. In this case, however, it stands for $a < b$ and $b < c$ which are both sensible on their own and together they imply $a < c$, so there really isn’t any room for confusion.]

2. Writing “$h \to +\infty$” makes no sense. This $\to$-notation always needs a partner, i.e. “$h \to +\infty$ as $x \to 0$” (where presumably $h$ would depend on $x$ in some way). A variable cannot go off to infinity on its own, it always needs to do so in response to some other change. If you want to include a formal symbol “$\infty$” in your set, that is fine.

3. Writing “$h \in \mathbb Z \geq 0 \vee {}\to + \infty$” makes even less sense. I think that you intend to say that $h$ is an integer greater than or equal $0$ or goes to infinity; however, unlike in English, “or” (i.e. “$\vee$”) can only connect complete statements which “goes to infinity” is not. In English, we essentially mentally insert a copy of the subject of the sentence (here “$h$”) after the “or”, in mathematical notation you need to be explicit. [There are only very few cases where something like this is allowed, most importantly when writing $n, h \in \mathbb Z$ instead of $n \in \mathbb Z \wedge h \in \mathbb Z$.]

4. You seem to want to use set-builder notation which always requires you to have two parts between $\{$ and $\}$: First, the variable you want to use in your description and second (after a “$|$” or “$:$”, or rarer, “$;$” or “$,$”), the conditions the elements need to fulfill. [For set-theoretic reasons that I don’t want to go into right now, the base set your objects come from is usually placed in the first part but not doing so is generally acceptable.] It is also okay to “destructure” your elements in the first part, so if you want to have a set of pairs, you can write $\{ (a, b) | \dots \}$ instead of $\{ x | x = (a, b), \dots \}$.

Putting all of these corrections together, we get $$ A = \{ (n, h) | n \in \mathbb Z \wedge (h \in \mathbb Z \vee h = +\infty) \} $$ or maybe $$ A = \mathbb Z^2 \cup \{ (n, +\infty) | n \in \mathbb Z \}. $$