Visual Studio App Center - Remove unprovisioned devices / re-sign manually?
As of today, there is no way to do either of things mentioned in my question. The AppCenter team is going to be looking at adding these features in the near future.
Follow the feature request here: https://github.com/microsoft/appcenter/issues/951