Windows: How do I remove a file's ACL, and just let it inherit the ACL on the containing directory?

You could use the reset function of icacls:

ICACLS "C:\path\to\folder" /reset /T /C

then enable inheritance:

ICACLS "C:\path\to\folder" /inheritance:e /T /C