What did Gauss think about infinity?
Solution 1:
Here is a blog post from R J Lipton which throws some light on this question. Quoting from a letter by Gauss:
... so protestiere ich zuvörderst gegen den Gebrauch einer unendlichen Größe als einer vollendeten, welcher in der Mathematik niemals erlaubt ist. Das Unendliche ist nur eine façon de parler, indem man eigentlich von Grenzen spricht, denen gewisse Verhältnisse so nahe kommen als man will, während anderen ohne Einschränkung zu wachsen gestattet ist.
The blog gives this translation:
... first of all I must protest against the use of an infinite magnitude as a completed quantity, which is never allowed in mathematics. The Infinite is just a mannner of speaking, in which one is really talking in terms of limits, which certain ratios may approach as close as one wishes, while others may be allowed to increase without restriction.
Solution 2:
Your interlocutor seems to oppose infinity (and attribute similar views to Gauss) on finitist or constructivist grounds. If this is the case, he would probably similarly oppose infinitesimals. This is because specifying an infinitesimal typically involves an infinite amount of data, at least in modern theories.
Here he would be wrong to assume similar beliefs on Gauss's part because Gauss specifically and routinely used infinitesimals in his development of differential geometry. A detailed discussion of this may be found in the book by Michael Spivak on Differential Geometry, Third edition, volume 2, chapter 4. The discussion starts on page 62 as follows: "Gauss now nonchalantly introduces infinitely small quantities..."
Your interlocutor also mentioned hierarchies of infinities. On page 75 in Spivak's translation of Gauss, one finds products of infinitesimals, and an expression for curvature in terms of these. These are second order infinitesimals. Thus Gauss dealt with a hierarchy of infinitesimals.
Solution 3:
You are right. Gauss did not believe in finished infinity. He would have condemned Cantor's ideas.
(Was nun Ihren Beweis anbelangt), „so protestiere ich zuvörderst gegen den Gebrauch einer unendlichen Größe als einer vollendeten, welcher in der Mathematik niemals erlaubt ist. Das Unendliche ist nur eine facon de parler, indem man eigentlich von Grenzen spricht, denen gewisse Verhältnisse so nahe kommen als man will, während anderen ohne Einschränkung zu wachsen gestattet ist. [C. F. Gauß, Briefwechsel mit Schumacher, Bd. II, p. 268 (1831)]
Translation from comments
"I protest against the use of infinite magnitude as something completed, which is never permissible in mathematics. Infinity is merely a way of speaking, the true meaning being a limit which certain ratios approach indefinitely close, while others are permitted to increase without restriction"