Can 'to' in 'to + verb' be an adverb?

The to in the full infinitive is considered a grammatical particle, i.e., a word that does not serve a part-of-speech function by itself, but only in combination with another word in the sentence.