Possessive apostrophe for song title ending in a exclamation and in quotes

Solution 1:

Setting the title in italics, which is not unusual, would solve the problem:

Press Heavenwards!’s exclamation mark gives away a Godspeed influence.

If setting in italics is not an option, perhaps because of your house style, then the quotation marks collide with the necessary apostrophe, as you've noted. It’s also likely that that sentence would be pronounced as heavenwards’ rather than heavenwards’s, which causes its own difficulties in adding ’s to the title.

One solution is to recast the sentence in order not to use the Saxon genitive:

The exclamation mark of ‘Press Heavenwards!’ gives away a Godspeed influence.
The exclamation mark in the title of ‘Press Heavenwards!’ gives away a Godspeed influence.