What is meant by a "structure map"?

Solution 1:

A structure map is a map which is being specified as part of a structure!

For example, as you say, a scheme over a scheme $S$ is a scheme $X$ equipped with a map $X \to S$; that map specifies the structure of being a scheme over $S$ and so is the structure map. Similarly, a limit or colimit is an object equipped with maps such that etc., and those maps specify the structure of being a limit or colimit and so are the structure maps.

Solution 2:

From any category $\mathcal C$ and any object $c$ of $\mathcal C$, you can construct a new category $\mathcal C_{ / c}$, called the slice category over $c$ or the category of objects over $c$, whose

• objects are the morphisms $p \colon c' \to c$ of $\mathcal C$ with codomain $c$,
• morphisms $(c' \overset p \to c) \to (c'' \overset q \to c)$ are those arrow $r \colon c' \to c''$ of $\mathcal C$ such that commutes $$ \begin{matrix} c' & \overset r \longrightarrow & c'' \\ \searrow \!\!\!\!\!\!\!\!\!\!\! & & \!\!\!\!\!\!\!\!\!\!\!\!\!\! \swarrow\\ & c, & \end{matrix}$$
• composition is induced by the one in $\mathcal C$.

For any object $c' \overset p \to c$ of $\mathcal C_{/c}$, one can call $p$ a structure map for $c'$.

With this definition, an $S$-scheme $X$ is an object $X\overset p \to S$ of the category $\mathsf{Sch}_{/S}$ ($\mathsf{Sch}$ being the category of scheme).

For a small category $\mathcal I$ and any category $\mathcal C$, the category of diagrams of shape $\mathcal I$ in $\mathcal C$ is the category $\mathcal C^{\mathcal I}$ whose objects are the functor $\mathcal I \to \mathcal C$ and the morphisms are the natural transformations between such functors. Take a diagram $D \colon \mathcal I \to \mathcal C$ and consider the full subcategory $\mathrm{Cone}(D)$ of $(\mathcal C^\mathcal I)_{/D}$ whose objects are those $D' \to D$ with $$ D'(i) = D'(j) \forall i,j\in\mathcal I \quad\text{and}\quad D'(i\to j)=\mathrm{id}_{D'(i)} \forall (i\to j) \in D' .$$ Then, by definition, the limit of $D$, if it exists, is the final object of $\mathrm{Cone}(D)$. It is an object $\lim_\mathcal I D \overset p \to D$ and p is the structure map of this limit.

You can formalize as well the structure map of a colimit by dualizing all the notion (coslice category ${}_{c\backslash}\mathcal C$, cocone, etc.).