"It did the job" meaning

Does it mean "it could be better but is ok" (small disappointment), or "it really worked!" (surprise and content)? Depends on the context?

Also, is there a difference between "That did the job" and "It did the job"?

You could use it either way - everything depends on the context, and your intonation when you say it. However, I'd say there's a slight bias toward "small disappointment" - especially if you preface it with "well": Q: I heard you ran out of deviled ham at your cocktail party, and served cat food instead. How did that work out for you? A: Well, it did the job... everybody went home.

As for the That/It distinction, you can use "That did the job!" in an enthusiastic, positive sense, whereas "It did the job" would sound odd except as the answer to a question, and (as I mentioned above) it would more likely be used to damn with faint praise.

It's context-dependent.

Negative:

Bill: What do you drive?

Bob: An old Honda Civic. It's not pretty, but it gets the job done.

Positive:

Bill: Did you install the new software update?

Bob: Yes, and it did the job! The program no longer crashes!