Exponentiation and a weak fragment of arithmetic

This is only a partial positive answer and it's far from optimal.

The statement is that if $\mathfrak{N}$ is a model of PA and $M\subseteq N$ is any cut (closed under successor) then $\mathfrak{M}$ (which as per your question we'll assume is closed under addition and multiplication) interprets $I\Delta_0 + \text{Exp}$ (using a parameter, although I think the parameter can be removed), where $\text{Exp}$ is the axiom that says that $x\mapsto 2^x$ is a total function. This is a slightly different question than the one you asked because on the one hand this is an interpretation, not an initial segment, but on the other hand we're not using the full structure $\mathfrak{N}$, just the cut. The reason why this might indicate that your actual question has a positive answer is that the proof that Robinson arithmetic does not interpret $I\Delta_0+\text{Exp}$ goes through the fact that $I\Delta_0+\text{Exp}$ proves the consistency of Robinson arithmetic, but any structure like $\mathfrak{M}$ satisfies all the $\Pi_1$ consequences of PA, and in particular the consistency of Robinson arithmetic, so that proof can't work to resolve your question.

The proof works in two cases, but these can be combined into a single first-order interpretation (with a parameter).

Note that $\mathfrak{M}\models I\Delta_0$, so in particular the graph of the function exponential function, $y = 2^x$ is a definable predicate in $\mathfrak{M}$ and it proves that the inductive properties of $2^x$, in particular if $2^x$ exists, then $2^{x+1}$ exists as well. Either $\mathfrak{M} \models \text{Exp}$ in which case we are done or by arguments in 'Interpretability in Robinson's Q' by Ferreira and Ferreira, there is a definable cut $C$ of $\mathfrak{M}$ such that $C \models I\Delta_0 + \Omega_2 + B\Sigma_1$, which also has a proper definable sub-cut $C^{\prime}$ such that for any $x\in C^{\prime}$, $2^{2^x}\in C$ and such that $C^{\prime}$ is closed under addition and multiplication. The point is that $I\Delta_0 + \Omega_2 + B\Sigma_1$ is enough to define the "$x$th binary bit of $y$" function and so we can code infinite subsets of $C^{\prime}$ in the binary expansions of numbers in $C$.

PA proves $\text{Con}(T)$, for any finite fragment $T$ of PA, including one big enough to prove that $x^y$ is a total function, so by the arithmetized completeness theorem there is a definable interpretation of $T$ in $\mathfrak{N}$, with constants for $0$ and $1$ and a function symbol for $x^y$. I claim that we can arrange this interpretation so that $0$ and $1$ are coded by standard numbers (i.e. $0$ and $1$) and that for any elements $x$ and $y$, the codes $\ulcorner x+y \urcorner$, $\ulcorner x\cdot y \urcorner$, and $\ulcorner x^y \urcorner$ are all less than or equal to $6 (\ulcorner x \urcorner + \ulcorner y \urcorner + 1)^2$ or so, all that matters is that it's a polynomial bound.

This follows from the fact that the standard Cantor pairing function is a polynomial, so the interpretation will be given by a definable predicate $I(x)$ and the functions $x+y$, $x\cdot y$, and $x^y$ will be interpreted by fixed polynomial expressions.

Now we can find a non-standard number $N\in C \setminus C^{\prime}$ such that $2^N \in C$ and we find a non-standard number $M \in C \setminus C^{\prime }$, with $M \leq 2^N - 1$, whose binary expansion corresponds to the first $N-1$ bits of $I(x)$. This gives us a parameter from which $C$ can define $C^\prime \cap I$, which by construction is closed under the polynomial expressions interpreting the functions $x+y$, $x\cdot y$, and $x^y$. Thus $C$ (and therefore $\mathfrak{M}$) interprets a structure closed under addition, multiplication, and exponentiation all obeying the correct inductive definitions and identities (as well as any finite collection of $\Pi_1$ consequences of PA that you want).