Difference between undecidable statements in set-theory and number theory?

Do all statements about the integers have a definite truth value? For instance: Goodstein's theorem is clearly true, otherwise we could find a finite counterexample thus it would be possible to disprove it. So the independence proof is also a proof. Godel's theorem only claims there are true statements that are unprovable.

What is it that makes statements about the integers have a clear true/false-value (by some higher-order reasoning), but statements about sets such as AC and CH dont?

Does the higher-order reasoning just become more and more diffuse (or subjective) or is there some fundamental leap between undecidable statements about the integers vs sets?

There is a lot to address here. You might be getting at any of the following points:

The existence of $\Pi_1^0$ statements: A statement is called $\Pi_1^0$ if, if it were false, it could be disproven by a finite amount of data. Goldbach's conjecture, "every even number is the sum of two primes", is $\Pi_1^0$, because to disprove it you just have to give an even number $2n$, and a nontrivial factor of $2n-p$ for each prime $p < 2n$. A $\Pi_1^0$ statement, if undecidable, is always true. See Wikipedia for the rigorous definition and much more.

It is far from the case that all interesting problems in number theory are $\Pi_1^0$. Consider the twin prime conjecture, "there are infinitely many pairs of primes which differ by $2$." This could be undecidable in two different ways. Maybe it is true, but we can't prove it. Or, maybe there are no twin primes past $10^{10^{10}}$, and we can't prove that. Either way, no finite amount of data will settle the issue.

Goodstein's theorem is not $\Pi_1^0$: Contrary to your statement, a finite amount of data cannot disprove Goodstein's theorem. Suppose that you knew that the Goodstein sequence starting at $100$ did not terminate. How could you convince me o this with any finite computation?

Now, in fact, Goodstein's theorem is true, because induction up to $\epsilon_0$ is valid. See this MO question for more. But consider the Collatz conjecture. Like Goodstein's theorem, it says that a recursively defined sequence starting at any integer eventually terminates. It is perfectly possible for Collatz to be either undecidable and true or undecidable and false.

ZFC does prove more than PA: The proof of Goodstein's theorem can be formalized in ZFC. So can the intuitive proof that PA is consistent, namely, "of course it's consistent, it is a set of true statements about $\mathbb{Z}$!" So it is true that being undecidable in ZFC is a lot stronger than being so in PA.

A point of philosophy: Some mathematicians will defend the belief that all statements about $\mathbb{Z}$ are either true or false (though they may be unprovable from the current axioms) but this need not be true about sets. The idea here is that there is only one set of integers, but there are many equally good versions of set theory. Scott Aaronson defends this view here; JDH defends the "more than one set theory" here. (I don't know whether or not JDH thinks there is more than one version of $\mathbb{Z}$.)

Note that this is much stronger than the claim that all $\Pi_1^0$ statements are true or false. Scott, for example, presumably believes that the twin prime conjecture is either true or false, even though no finite amount of computation could ever settle it.

I have not thought enough about this question to form an opinion; it is ultimately a matter of philosophy, not math.

What is it that makes statements about the integers have a clear true/false-value (by some higher-order reasoning), but statements about sets such as AC and CH dont?

The key difference is that

• The set of natural numbers is defined by a sort of "minimailty principle": the natural numbers are, up to isomorphism, the smallest set containing 0 and closed under the function $f(x) = x+1$.

• The collection of all sets is defined by a sort of "maximality principle": it consists of all sets.

This difference makes the natural numbers concrete in a certain way the the collection of all sets is not.

If we try to take the corresponding "minimality" principle for sets, we end up with the constructible universe $L$, or with some sort of minimal submodel of it, for which we can answer many of the questions like AC and CH that are independent of ZFC.