OS X Server: Can't upgrade or configure: App Store fails: “This action could not be completed”

Solution 1:

It looks like you've done more than the expected basic troubleshooting (reboot, try debug mode, single user, try the same AppleID from a new user account) and you're looking at either a cascading failure (multiple caches or multiple keychains with bad signing data) or some lower system level failure in the code signing / update / permissions.

If you bring a second Apple ID to the app store app on this Mac and you can't sign in with two accounts - the conclusion would be that the OS is the problem.

If you can use the Apple ID you want on this Mac on another Mac to sign in to the store - you can rule out network issues and a problem with the intended Apple ID.

I would probably download the Server app from a new OS (you could boot to recovery and install OS X cleanly onto a USB or other external drive and use the Mini or you could use a different Mac entirely. Once you have the correct Server downloaded - transfer it to the problematic Mac and see if it can launch.

At that point, I'd probably get a good backup of the server and consider an outage to nuke and pave:

1. The wikiadmin command is beefed up recently - https://support.apple.com/en-us/HT206232
2. Bender will make backups of OD, complete settings backup as well as individual silo backup files so you can selectively restore each service one by one (which in your case might be of benefit).
3. Time Machine - just in case you contact Apple engineering for support with this case. If you have AppleCare on the Mac in question - enterprise support is a phone call away, and that should be your first call rather than doing anything I suggest here.
4. Bootable clone backup - using Disk Utility, Carbon Copy Cloner, SuperDuper or similar.

If you really want to dig in, maybe run the latest combo installer, but in truth - server app is running all the services in a "pause/wait/hold" pattern since the OS updated and the server needs to run all the migration steps - so most of the help in the net isn't geared for a system in your state.