Looking for a rigorous treatment of improper multiple Riemann integrals
Munkres Analysis on Manifolds covers this in Chapter 3.15. Having now twice taught from Munkres textbook, I will say that this is the section I find the hardest. I constantly find myself at the chalk board saying "oh, and now we'll need another $\epsilon$" and "oh, I forgot to put take an exhaustion by compact subsets five minutes ago." Integrals of bounded functions on bounded domains is really clean and students follow it well, and when we hit the improper integrals is where the confusion sets in.
Hubbard and Hubbard Vector calculus, linear algebra, and differential forms covers this in Chapter 4.11. (Chapter numbers vary from edition to edition; the chapter title is "Improper Integrals".) When I read this a few years ago I thought "This is so much clearer! I am definitely using this in place of Munkres next time!" But next time hasn't come yet, so I can't say if the students agree.