Usage of contractions like "it's" and "that's" in textbooks

Is it considered bad style to use abbreviations contractions like "it's" and "that's" (instead of spelling them out as "it is" and "that is") in a textbook or academic publication?

It depends on the context. "[A] textbook or academic publication" includes a wide range of publications, including textbooks for grade school, where I can see that's being used (perhaps not in running text, but at least in a sidebar). Even published journal papers in some fields may include such words: mathematics papers are not known for their careful attention to formal writing, and may well include it's and that's. (It will depend on the journal, however, and, of course, the authors.) But as an overly general rule, yes, it's considered bad style to use them in textbooks and academic publications.

Yes, that's what I've always been taught. Abbreviations are more informal / oral speech.