Solution 1:

Here's an attempt (could be mistakes so be wary!).

We have $$ M \otimes_{A[[t]]}^{\mathbb{L}} B[[t]] \cong (M \otimes_{A[[t]]}^{\mathbb{L}} A[[t]]/t^n) \otimes_{A[[t]]}^{\mathbb{L}} B[[t]]. $$

By associativity this is (quasi-isomorphic) to $$ M \otimes_{A[[t]]}^{\mathbb{L}} B[[t]]/t^n. $$ Since $A[[t]] \rightarrow A[[t]]/t^n$ is surjective, this is the same as $$ M \otimes_{A[[t]]/t^n}^{\mathbb{L}} B[[t]]/t^n.$$

And this satisfies the conclusion you want because of your conditions on $M$.