Windows explorer keeps handle open on executable files
Solution 1:
As weird as it may sound (I don't really understand) but following the steps described in this blog post corrected everything.
Basically I typed the following in an administrator Command Prompt and it all started to work magically:
net user administrator /active:yes