Separated schemes and unicity of extension

Solution 1:

To answer your question (1): Let $X=\mathrm{Spec}(B)$ be such that $B\to O_X(U)$ is not injective for some dense open subset $U$ (this can't happen if $B$ is reduced), let $Y=\mathrm{Spec}\mathbb Z[t]$. Fix an element $b\in B$ non-zero such that $b|_U=0$.

Let $\varphi: \mathbb Z[t] \to B$ be any ring homomorphism and let $\psi : \mathbb Z[t] \to B$ be defined by $\psi(t)=\varphi(t)+b$. Then the corresponding morphisms $f, g : X\to Y$ coincide on $U$ but are not equal.

Standard example of such $B$: $B=k[x,y]/(x^2, xy)$ and $U=X\setminus \{ (0,0)\}=D(y)$.

For your question (1), you can replace the hypothesis $X$ reduced by $S\to X$ schematically dominant. When $X$ is noetherian, this means that the image of $S\to X$ contains the associated points of $X$ (= maximal points of $X$ if the latter is reduced).