Position of prepositions in questions and clauses
The 'rule' for prepositions is that they should be placed before their complement, hence the name - pre-position. Failure to do this (and the 'never end a sentence with a preposition' rule...) is called preposition stranding.
Having said that, the 'rule' that prepositions should preceded their complement has no real basis, and was probably made up by John Dryden when he grumbled about Ben Jonson writing "the bodies that those souls were frightened from".
Maybe because of this, preposition stranding in English is very common:
in Wh- questions, because the preposition complement is the wh-, which has been fronted to make a question - "He talked about what?" > "What did he talk about?"
in relative clauses, again because the preposition complement gets moved - "I told you about this restaurant" > "This restaurant is the one [that] I told you about"
in passives/false passives - "He has slept in this bed" > "This bed has been slept in."
In informal English, the Prepositions are normally placed after the Verb - whether in Relative Clauses or questions.
From your example, "He's the person I spoke TO."
Also, we don't often use "whom" in spoken English. We use "who" or "that" or omit the Relative Pronoun altogether.
"He's the person I spoke to."