Executing bundled shell command in a hardened runtime application
The issue was related to my build process itself, not the hardened runtime which I suspected. In fact, when I enabled the hardened runtime, also a launcher application was put into place inside the bundle which starts the main application using the bundled JRE.
As this is the AdoptOpenJDK I mentioned, it uses spawn for Runtime.exec which caused the issue. Once I added System.setProperty( "jdk.lang.Process.launchMechanism", "FORK"); to my main method, Runtime.exec was working just fine again afterwards, even with the hardened runtime enabled.