Gerund? Infinitive? Why, when we talk about jobs, do we say "I have a job taking people on tours" instead of "I have a job to take people on tours"?

Re:

My job is to drive people around the city.
My job is driving people around the city.

You now have a verb (is) in between your noun (job) and the descriptor. After "is," either the present participle or infinitive is correct.

Re:

Mandela gave him a job taking visitors around the prison.

The object of the sentence is "job," being acted on by Mandela. However the visitors are also objects of a verb, and this is what happens when you have a verb with an object modifying a word that is itself an object - we use the present participle ("taking . . . around" forming the present participle of the verb "to take around").

Notes:

• the answer does not depend on what Mandela did. It could be that Mandela fired him from a job taking..., or asked to talk to him about his job taking..., and the job would always be described as "taking visitors around the prison." Indeed it does not need Mandela at all. It could say: Joe has a job taking..., or I know of a job taking..., etc.

• the answer does not depend on the verb being used as to describe the job. The job could be taking visitors around the prison, or teaching English, or juggling chainsaws. It would always be "a job [x]ing [whatever]."

• incidentally, as the examples above show, it does not have to be a compound verb like "to take around." E.g., Mandela gave him a job teaching English.

• it does not have to be a "job." Mandela promised him a career, a position, an appointment, a vocation, a thing to do, etc. It will always be "a [w] [x]ing [whatever]. Indeed, Mandela might also show him a man carrying a box around the prison.

This is subtly different from something like:

Mandela showed him a dog chasing cats.

In this case, the words "chasing cats" describe what the dog was doing. We could add "that was" very easily. But we cannot add "that was" to your original sentence, because it would not be an accurate description of the relationship between the object and the noun phrase. "Mandela gave him a job that was taking people around the prison" sounds like the job is doing the taking. Rather, as said in a comment, the relationship would be made explicit with the words "consisting of" or something like that. Nonetheless, it is the object-verb-object ordering that results in the use of the -ing form.