Lebesgue Integral of Non-Measurable Function

In addition to the previous advice, note that the function you gave does not "approximate" $f$. An approximating sequence $\{s_n\}$ for $f$ (which $f$ would have iff $f$ were measurable provided the measure space for the domain of $f$ is complete) would need to be within a distance of $\epsilon$ from $f$ for any given $\epsilon >0$. However, the function $s$ you gave as an "approximation" cannot approximate $f$, in the sense that the sequence $\{s_n\}$ with $s_n=s$ $ \forall n \in \mathbb{N}$ gets no closer than $1$ from $\chi_{[0,1]}$.

Again, it is important to note that a function $f$ on a complete measure space $X$ is measurable if and only if $f$ is the pointwise limit of some sequence of simple functions -or, trivially, is a simple function itself. Consequently, any sequence of "simple" functions approximating a nonmeasurable function must contain a "simple" function with the characteristic function of a nonmeasurable set as part of its construction. In a sense, this is why you must include the assumption that $f$ is measurable in your definition for the Lebesgue integral.

To wit, recall that the integral of a characteristic function is the measure of the pullback set in your domain; in the case of your $f$, since $f$ is characteristic, the integral, were it to be defined, is the measure of the pullback, $$\int f d\mu = \mu\{f^{-1}\{1\}\}=\mu\{[0,1]\},$$

but you have not defined the measure for $[0,1]$, which is not even in your $\sigma$-algebra; nor can we infer the measure of $[0,1]$ from the definition you gave for your measure space, as your collection of measurable subsets is already a closed $\sigma$-algebra that is $\sigma$-finite under $\mu$ (hence, your measure space cannot even be extended, in the usual Caratheodory way, to include $[0,1]$ with an accompanying well-defined measurement).

As a curiosity tangential to your question, it is possible for nonmeasurable functions to arise from limits of simple functions in a complete measure space, but such a collection of simple functions must be uncountable. For instance, with Lebesgue measure on $\mathbb{R}$, take $f=\chi_V$ to be the characteristic function of the (uncountable and nonmeasurable) Vitali set $V$ on $[0,1]$, and consider the (uncountable) collection of measurable functions $\{\chi_v\}, v\in V$. Then $\chi_V=\sup \{\chi_v\}_{v\in V}$. Were we to define an integral as you wish, then in this case, you may want to say that the integral $\int \chi_V = \sup \{ \int \chi_v \} = 0$. But, again, since $\chi_V$ is itself characteristic, we should then have $\mu(V)=0$, but $\mu(V)$ is not defined for $V$ under the Lebesgue measure.

To address your request for a resource, see Royden's Real Analysis, 4th ed., chapters 17 and 18 (particularly pp. 362-363 were helpful as a reference to me for this post).


Also, maybe it is useful to show the bad properties of this lower integral when applied to non-measurable funtions. Take your $\{\varnothing, \mathbb R\}$ example. Let $f = \chi_{[0,1]}$ and $g = 1-f$. Then $\int f = \int g = 0$ but $\int(f+g) = 1$. So even simple linearity fails.