Does "the alphabet of the language of propositional logic" have no function symbols, relation symbols, and constants?
Exactly. Propositional logic has only propositional variables and connectives (and parentheses).
See page 201, Def.4.1.
You may have different languages according to the set of basic connectives used.