Using "and" twice for four items
Solution 1:
While your construction This book treats single and multivariable differential and integral calculus needs to be read a couple of times to get the meaning, I think that is because of the topic, not the construction. It is a concise manner of expressing the idea.
Solution 2:
You're addressing a bit of a technical audience who may well understand your sentence as written, but it is ambiguous. Sometimes there isn't a great way to succinctly express a combinatoric set of things like this. Sometimes you can reword it in a domain-specific way as @Josh61 does in the comment. Sometimes for full clarity you just need to say it the long way, like you did in the question:
The book discusses single-variable differential calculus, single-variable integral calculus, multivariable differential calculus and multivariable integral calculus.
Solution 3:
I agree with @Kris that it is gramatically correct and unambiguous, but doesn't read very well. How's about using a comma and the word both
in the right place?
This book treats differential and integral calculus, both single- and multivariable.
You could even omit both
without losing meaning, and lose only a bit of readability.