What is the opposite of gem-like? [closed]

I'm going to suggest "banal" as an answer to this question. A banal person would be someone who generally makes one feel worse for time in their company. If you're looking for a word which specifically conveys lack of value, "worthless" is also a possibility.

Rough-like or just plain rough.

a : something in a crude, unfinished, or preliminary state

See also: a diamond in the rough.

EDIT: After reading OP's edits I thought to look up boorish. While that doesn't seem to fit exactly, I thought this synonym discussion on the page was interesting:

boorish, churlish, loutish, clownish mean uncouth in manners or appearance.

boorish implies rudeness of manner due to insensitiveness to others' feelings and unwillingness to be agreeable .

churlish suggests surliness, unresponsiveness, and ungraciousness .

loutish implies bodily awkwardness together with stupidity .

clownish suggests ill-bred awkwardness, ignorance or stupidity, ungainliness, and often a propensity for absurd antics .