You need to install a package for that. D'oh!
Running:
sudo apt-get install libapache2-mod-php
resolved the problem.