Meaning of $\int\mathop{}\!\mathrm{d}^4x$
It is a notation usually used in physics.
In their language $$ \int\mathop{}\!\mathrm{d}^3 p \,f(p) = \int_{\Omega} f(x,y,z)\mathop{}\!\mathrm{d}x\mathop{}\!\mathrm{d}y\mathop{}\!\mathrm{d}z. $$ Several remarks:
The domain where the integration is performed is unspecified, because we can have the flexibility of doing integration in momentum space or position space.
The exponent on the shoulder of $\mathrm{d}$ specifies the dimensionality of this integral, so no $\iint$ or $\iiint$.
The variable the integration is performed on is written before the integrand.
So if it is $\mathrm{d}^4 x $, normally it means to integrate in the whole space-time.
Some Updates: Not an expert in GR but if the integration is $$ S=\frac{c^4}{16\pi G}\int\mathop{}\!\mathrm{d}^4x \, R \sqrt{-g}, $$ then $\mathrm{d}^4x\,\sqrt{-g}$ means the integration is done in the whole space-time $\mathcal{M}$ with certain metric $g$. The metric also has a negative determinant (relevant to the $(− + + +)$ sign convention). The "volume" element in space-time for this integration is actually $$ \underbrace{\mathrm{d}^4 x\,\sqrt{-g}}_{\text{Physics type of notation}} = \underbrace{\sqrt{|g|} d x\wedge d y\wedge d z\wedge cd t}_{\text{Volume element of integration performed on a manifold with metric } g}. $$ Here we can't separately write $\mathrm{d}^4 x$, it would cause certain level of confusion of the metric. Also the mathematical type of notation is only true whenever we equip the manifold locally with a coordinate system $(dx,dy,dz,-cdt)$ while the physics's notation leaves the volume element's coordinate system unspecified to avoid some technique issues.
Now because the space-time's metric $g$ is not a constant but rather changing in the whole space-time, we normally we do not write in the form of $\mathrm{d} x\,\mathrm{d} y\,\mathrm{d} z\,c\mathrm{d} t$ or $ \mathrm{d} x\wedge\mathrm{d} y\wedge\mathrm{d} z\wedge c\mathrm{d} t$ unless we know the space-time is flat (Minkowski space-time). I suggest you try to derive the action using a unit mass particle moving in a gravitational field in Minkowski time yourself to better understand the notation.
A simple analogy: The "area" element in polar coordinate system is $rdrd\theta$ (written in math notation convention), while in physics the integration is written as: $$ \int\mathop{}\!\mathrm{d} \theta\mathrm{d} r\,r \,f(r,\theta). $$
This is particularly common in Physics where it represents an integral over all spacetime coordinates. Here $x=(t,x,y,z)$ in natural units (note the $x$ on the left hand side isn't the same as the one on the right hand side. Then $$\int \mathrm{d}^4x \equiv \int \mathrm{d}t \int \mathrm{d}x \int \mathrm{d}y \int \mathrm{d}z$$ is just a shorthand.
Note also that in Physics, we write the $\mathrm{d}x$ before the integrand, not after.
When we work with function $f : \Bbb R \to \Bbb R$ it's common to denote the integral of $f$ over $[a,b]$ as simply:
$$\int_a^bf(x)dx$$
This notation has much to do with the fact that in classical treatment, $dx$ was thought of as an "infinitesimal lenght" in the $x$ axis, $f(x)dx$ an infinitesimal area, and the integral the sum of those infinitesimal areas. This idea of infinitesimals lead to confusions, so that it was swept away so that we have today the modern and formal standard analysis. In this case, the $dx$ is usually superfluous and we write the integral simply as:
$$\int_a^bf$$
Now, for functions $f: A \subset \Bbb R^n \to \Bbb R$ we usually write the integral exactly as above or in the "classical language" simply as:
$$\int_A f = \idotsint_Af(x_1,\dots,x_n)dx_1\cdots dx_n$$
So people usually simplify writting $d^n x$ so that we have:
$$\int_A f = \idotsint_A f(x) d^nx$$
Now, this is just notation. Rigorous meaning, however can be given in the context of differential forms. In truth, when we study analysis over $\Bbb R^n$ and when we study differential geometry, we realize that the really meaningful objects to integrate are the so called differential forms. In that context, $dx$, $dy$ and etc receive a formal treatment: they are differential forms.
There we introduce the concept of a wedge product between forms denoted $\wedge$ and so people can usually write:
$$d^n x = dx^1 \wedge \cdots \wedge dx^n$$
Where $dx^i$ is the differential of the $i$-th coordinate function. It turns out that this is a "volume form", then scalar functions can be put into correspondence with $n$-forms in $n$-space, so that for $f : \Bbb R^n \to \Bbb R$ we associate the differential form $\omega \in \Omega^n(\Bbb R^n)$ given by :
$$\omega = f d^n x = f dx^1 \wedge \cdots \wedge dx^n$$
To see more about differential forms look at Spivak's Calculus on Manifolds. I hope this helps you somehow. Good luck!