After a failed Azure Deploy, our deployed AppService is empty
Solution 1:
This is expected. Your pipeline is doing two things, deploying the app service, then deploying your app. If the first part succeeds it deploys an app service with default files, if your second step fails then it will be left in that state.
This should only happen on first deployment. If you run a subsequent deployment where the first step succeeds it will not overwrite the existing files in the web app, it will leave them as is. If your second step fails then the old files should still be there, unless your second step failed part way through and overwrote some of the files.
If you want to ensure you have an option to roll back to the previous version even if the files do get overwritten then you should look at using deployment slots.