“To agree with someone”: is that prepositional phrase an adverbial or a nominal one?
"With" is always a preposition (http://partofspeech.org/what-part-of-speech-is-with/). Adverbs answer one of these: how, when, where, why, to what extent, how often or how much.