Powerpoint 2010 types the wrong character

These are smart quotes and will display based on the language chosen. Double angle quotes are used used by some languages, like French or Italian.

To avoid the double angle quotes, turn off smart quotes via File > Options > Proofing > Autocorrect Options > Straight quotes with smart quotes

Smart quotes are rendered differently, depending on the language settings. In French and Italian, they will turn the double quote into the double angle quote. In English, they will turn the quote into curly quotes (different for opening and closing quotes).

If you set a language and then use smart quotes, the smart quotes for that language will be applied. If you then change the language of that text to a different language, the characters still remain and need to be changed manually or with find and replace.

If you turn smart quotes off before entering your text, the language does not matter, since smart quotes will not be used and double quotes will not be replaced with a language specific character.

The concept of smart quotes is that the straight double quote sign " will be replaced by a different character, one for the opening quote and one for the closing quote. Here are some examples for the quotes in different languages.

“English”       147 148
« French »      171 187
«Italian»       171 187
„German“        132 147
"Straight"      34  34

Note how the French quotes differ from the Italian ones, since they have an additional space character after the opening and before the closing quote. Next to each language you can see the ascii code for each of the characters.

Also note that the "straight" example has the same code for opening and closing quote. This is where the "smart" bit comes in. PowerPoint (and Word) recognize the opening and closing quotes and will replace them with different characters, depending on the conventions of that language. This replacement will only happen while typing (it is, after all, an auto-correct feature), so changing the language after the text has been typed will have no effect.

Those are french or maybe italian (thanks teylyn) quotes. Verify that your document language is set correctly, since you seem to have checked your system locale already.

Edit: You can see the documents language in the status bar on the bottom of the Powerpoint 2010 Window.