Limits and substitution

I'm afraid this isn't quite what you're after, but it seems to be a decent starting point. These conditions, while restrictive, appear to be necessary for the following proof.

Assume $\lim_{x\to a}g(x)=b$, where the function $g$ also satisfies: $$ \text{there exists a neighborhood $U$ of $a$ such that $b\not\in g(U\setminus\{a\})$} \tag{1}$$ and for each $(y_n)$ converging to $b$ with $y_n\ne b$, $$ \text{there exists $(x_n)$ such that $g(x_n)=y_n$ for large $n$ and $x_n\to a$}. \tag{2}$$

Then we show that $$\lim_{x\to a}f(g(a)) = \lim_{y\to b}f(y).$$

Proof:

Let $L_1:=\lim_{x\to a}f(g(a))$ and $L_2:=\lim_{y\to b}f(y)$, either of which may or may not exist. We will use the sequential criterion.

First suppose $L_2$ exists and let $(x_n)$ be a sequence such that $x_n\to a$ and $x_n\ne a$ for all $n$. Since $\lim_{x\to a}g(x)=b$, we have $g(x_n)\to b$. Due to $(1)$, we have $g(x_n)\ne b$ for $n$ large enough. Then, because $L_2$ exists, we have $f(g(x_n))\to L_2$. This proves by the sequential criterion that $L_1$ exists and $L_1=L_2$.

Now assume $L_1$ exists and let $(y_n)$ be a sequence such that $y_n\to b$ and $y_n\ne b$ for all $n$. By $(2)$ there exists a sequence $(x_n)$ such that $g(x_n)=y_n$ for large $n$ and $x_n\to a$. Since $y_n\ne b$ for all $n$, we have $x_n\ne a$ for large $n$. Then $$ \lim_{n\to\infty}f(y_n)=\lim_{n\to\infty}f(g(x_n))=L_1, $$ and so the sequential criterion implies $L_2$ exists and $L_2=L_1$.