“based in” vs “based out of ”

They're mostly the same. "Based out of" often suggests that the subject maintains a headquarters or home office in the given location, but spends a majority or other significant amount of time working in other locations; "based in" suggests that the subject works in the given location most of the time. But counterexamples are common, so you shouldn't make assumptions based solely on the wording.

Except in the military context, “based out of ” is dubious, confusing, and poor English. I just saw it cause a very expensive confusion in a formal tax analysis from an American tax expert.

He wrote that Company X was based out of Country Y, and all (foreign) readers of the report interpreted that Company X was based outside Country Y, and as such had no taxable presence there.

This led to erroneous decisions being taken that resulted in payment of millions of U.S. dollars of taxes!

"Based in" implies the major operations of a business or entity is contained wholly or primarily in that city. "Based out of" implies that though the "home" of the business may be there, the operations of that business take place in other places as well.

"Based out of" is a common term to refer to the home base of a military unit: the 101st Airborne is "based out of" Fort Campbell, Kentucky, but they're currently getting it done in Afghanistan. We don't usually say "based in", because unfortunately, soldiers don't get to wake up in bed next to their spouses, have a nice breakfast and then commute to war.

in which situation is each better used?

That would seem to be in situations where American English is used. I have heard "based out of" used by Americans in speech and writing but, not by anyone else. It's not used in the UK and certainly sounds odd and weird, from my British perspective.

I’ve never until now seen based out of. It seems to represent confusion between based in and working out of.