"Iterate" and "iteration" as nouns

What are the differences between iterate and iteration as nouns?

I don't quite understand the definition of iterate as noun:

A quantity arrived at by iteration

For example, in computer programming, there are language features for iterations, such as for loop, while loop. Which one is correct: each iterate or each iteration?

I saw a lot of scientific literature use each iterate, such as this one:

Since each iterate is contained within an interval ...

Each iteration is correct here.

Iterate as a noun means the end result of many iterations. So in an iterative function, each iteration is one loop of calculation, whereas the iterate is the sum of all iterations performed.

An important thing to note here is that iterate can also mean a function that iterates. So if you ever see the noun iterate used, test both definitions to see which makes more sense.

In optimization, I think that it's common to use "iterate (noun)" to denote the variable vector at the current iteration. For example, when you try to minimize the function f(x) by iterative methods, you generate a sequence of iterates x_1,x_2,...,x_\infty that ideally converges to a minimum of f(x).

In this context, "iteration" is used to denote the full process of going from x_k to x_{k+1} and can include multiple calculations, evaluations, etc. The points x_k and x_{k+1} themselves are the iterates.

Example sentence: "For a problem having n variables, our algorithm has been proven to generate a sequence of iterates with the distance between the k-th iterate and the minimum x* bounded by |x_k - x* | < B*n/k, with B > 0 constant".

I've personally used "iterate" and "iteration" with this distinction many times in papers I've written, and other authors in my field appear to as well.

As nouns, in many contexts, iterate and iteration are synonymous.

The definition given by Wiktionary for iterate (noun) encompasses not just a quantity, but a mathematical concept:

(mathematics) a function that iterates
f²(x₀) is the second iterate of x₀ under f.

(PlanetMath has a more detailed mathematical description; they show the nouns iterate and iteration as synonyms for this idea.)

Each iterate refers to each subsequent result of iteration (x_k+1, in the paper you mentioned).

So, if an iterate is the result of iteration, what is an iteration?

Wiktionary tells us that it can mean a single repetition, which is what makes the two nearly synonymous. This could be used in math, computer science, or indeed in art or music or writing, meaning:

A variation of a design.

Iteration also means the act or use of repetition, and it has a domain-specific meaning you alluded to:

(computing) the use of repetition in a computer program, especially in the form of a loop

So a computing iteration sounds a lot like a mathematical iterate, except that iteration is much more common; I'll certainly say:

Hmm. It failed on the fourth iteration.
The iteration in my for loop isn't working!

An iterate can also be the state part way through an iteration. For example, in an infinite iteration that converges, you can say that the iteration can be stopped when the distance between the iterates x_n and x_{n+1} drops below a chosen threshold.