Term for a former prisoner, who seems to have an aura of jail hanging around him

I read in a book about this effect, and I just can't remember the term for it. It describes the effect with a name, something like: "shadow of jail about him" or something. This is seriously driving me crazy. I've tried and tried to remembered, and googled, and I still can't remember it.

It describes people who have got out of jail, and they seem to have the aura of jail still around them. I know it is actually a phrase, not something the author coined. Can someone please help me? And sorry for the vague kind of guidelines.

If you want to know, that book was something I read years ago when I was younger, called

Jo's Boys

There are a few metaphors that occur in Jo's Boys that resemble what you're talking about.

pp. 263: "They'd see and smell and feel the prison taint on me..."

pp. 327: "He longed to go home, but waited week after week to get the prison taint off him and the haggard look out of his face."

pp. 338: "All the old prison gloom seemed to settle like a black cloud on Dan's face..."

pp. 343: "... and to help wear away the first sharpness of the prison brand."

A person who has done time is often called a "jailbird", an "ex-con", an "old lag"...