I cannot understand the meaning of the following sentence fron Dickens' Notes of America [closed]

The last sentence from the following paragraph from Dickens is ambiguious for me; "He was only twenty-five years old, he said, and had grown recently, for it had been found necessary to make an addition to the legs of his inexpressibles. At fifteen he was a short boy, and in those days his English father and his Irish mother had rather snubbed him, as being too small of stature to sustain the credit of the family. He added that his health had not been good, though it was better now; but short people are not wanting who whisper that he drinks too hard "

Want here is used in the sense of be a lack of. There is no lack of short people who say privately that the young man drinks too much. I don't know whether Dickens is making some connection between alcoholism and a late growth spurt!

NB Inexpressibles is a jokey euphemism for trousers.