A fancier way of saying, “I'm not a fan of…”

I am looking for synonyms–the more ‘fanciful’ the better–of "not a fan" as in

• I'm not a fan of his, but give the man a break!

The expression, “I'm not a fan of his/her” or “I'm no fan of his/hers”, is often used when someone means they dislike a particular person, their views, work or their policies. More often than not, the speaker uses it in a non-derogatory fashion.

• ... I've really never been a fan of hers. I never disliked her, but was rather ambivalent.

• All Jenner said was, "I'm out of the loop, I can't say much". I'm not a fan of hers but what's the harm here?

• “I probably regret going after John McCain as hard as I did on a different format. I'm not a fan of his, but at times I made it more personal than it should be,…”

• I'm certainly no fan of his myself, but I definitely sympathize with him and shudder to imagine such a horrible end.

I would like users to suggest a more formal way of saying, I'm not a fan of ___. For example,

I'm ________ Jacob Rees-Mogg, the British Conservative MP, but I have to admit his English is flawless.

The higher and more intellectual the register, the subtler and less declamatory or assertive the tone generally becomes. In the very highest register, nuanced understatement becomes perhaps the best tool.

Compare the clamor of a frequently misquoted line from Mark Twain,

Reports of my death have been greatly exaggerated!

with his actual statement:

The report of my death was an exaggeration.

Twain used subtlety and understatement to make a funnier and more powerful statement. Through a kind of verbal jiu-jitsu he mocks the news reports (he had worked in journalism himself) while making fun of his own age and circumstances.

I would recommend you similarly use understatement to express your lack of approval for someone. To do so, you might have to abandon your "I'm not a _____ of so-and-so" construction to use something a bit less direct and a bit more polished. Using litotes can help you here as well.

I tend to place Mr. So-and-so beneath a pedestal, but ...
Mr. So-and-so is not really my idol, but ...
While perhaps no paragon himself, Mr. So-and-so ...

And so on.

My favorite expression to use in this case would be I don't particularly care for, as in:

I don't particularly care for Moby's style of Electronica, but it's undeniable that his single Porcelain was very well received around the world.

I hold no brief for...

_British
Not support or argue in favour of.
Oxford Living Dictionaries

I'm not enamored of...

_{(US enamor)}
1.1 Have a liking or admiration for.