Always show markup toolbar in Preview

Solution 1:

In addition to the already mentioned Shortcut ++A Preview in El Capitan (and Yosemite I guess) features a toolbar icon that allows to open the markup toolbar with a single click instead of clicking through the View menu:

Markup Toolbar Icon

To make the Toolbar visible, go to View > Show Toolbar in the menu. (Thanks to Rachel for this aditional hint!)

Solution 2:

No, the toolbar seems to reset every time a new image is opened.

However you can use the shortcut ++A to toggle the option to show/hide it.