Only keep the escape key on touch bar keyboard

I would like to remove all the keys on the touch bar except the escape key.

System Preferences -> Keyboard ->
  Touch Bar shows: Quick Actions
  Press Fn key to: Show Control Strip

After that you only have Esc and Configure in: Extension Preferences buttons on the Touch Bar, while quick actions are accessible through pressing the hardware Fn key.

Now, the question is how to also remove that Configure in: Extension Preferences button as it still results in accidental key presses, but at least it only shows a preferences window, which is easy to quickly close with Cmd+W.

System Preferences -> Keyboard -> Touch Bar shows: Expanded Control Strip

After that Customise Control Strip... and drag out all buttons from touch bar.