Can I get (the new) bash on Windows 10 to load .profile

You can change Bash shortcut to make it execute your .profile. Click on properties and append " --login" after the "~" in Target.

Note: This is the properties of the bash file, not the properties that show up in the upper left window of the bash shell. From the start menu, right click on "Bash on Ubuntu on Windows" and select More, then open File location. Right click on that file in the File Explorer and select properties. There's where you append " --login".

By default, this release of ubuntu-on-windows loads the file named ~/.bashrc (instead).

It's interesting though because ~/.profile has a section in there to automatically load ~/.bashrc. So if you reverse it to make .bashrc load .profile, you'll need to uncomment that.