Strong Markov property - Durrett

Markov chains are irrelevant here hence let us concentrate on the strong Markov property. Start from any process $(Z_n)_{n\geqslant0}$ (this is just a collection of random variables defined on the same probability space) and call $F_n$ the sigma-algebra generated by $(Z_k)_{k\leqslant n}$.

Then, the simple Markov property is to ask that $$ \mathrm E(\varphi(Z_n,Z_{n+1},Z_{n+2},\ldots)\mid F_n)=\mathrm E(\varphi(Z_n,Z_{n+1},Z_{n+2},\ldots)\mid X_n), $$ almost surely, for every $n\geqslant0$ and for every (for example) bounded measurable function $\varphi$.

Likewise, the strong Markov property is to ask that $$ \mathrm E(\varphi(Z_T,Z_{T+1},Z_{T+2},\ldots)\mid F_T)=\mathrm E(\varphi(Z_T,Z_{T+1},Z_{T+2},\ldots)\mid X_T), $$ almost surely on the event $[T\lt\infty]$, for every (for example) bounded measurable function $\varphi$ and for every stopping time $T$. (At this point, I assume you know what a stopping time $T$ is and what the sigma-algebra $F_T$ generated by a stopping time $T$ is.)

Since constants are stopping times, the strong Markov property implies the simple Markov property. For processes in discrete time, the two notions coincide (but be warned that this is not the case anymore for processes in continuous time).