SSH config: default password, except as specified
Solution 1:
From man 5 ssh_config:
Since the first obtained value for each parameter is used, more host-specific declarations should be given near the beginning of the file, and general defaults at the end.
Your Host * section is most general. It should be at the very end.