Cannot Eject USB Drive With NTFS and Bitlocker

I have an external HDD (WD My Passport Ultra) formatted with NTFS and encrypted with Bitlocker (Windows 10). After plugging in the drive, I can no longer remove the drive using "Eject". The error is "Windows is unable to stop the device 'USB Mass Storage Device'. Don't remove this device while it is still in use. Close any programs using this device and then remove it."

If I view the opened files using Process Explorer (search for "J:"), I see the files in use are:

  • J:\System Volume Information\FVE2.{...}
  • J:\System Volume Information\FVE2.{...}.1
  • J:\System Volume Information\FVE2.{...}.2
  • J:\$Extend\$RmMetadata\$TxfLog\$TxfLogContainer0000000000001
  • etc..

The reason I think this question is different from previous (eject USB) questions is that the FVE2 files seem to be specific to Bitlocker.

My question is, what is causing this problem, and what is the proper way to eject this external HDD without third-party programs that force-close all file handles? Since this is an encrypted drive, I'm extra paranoid about force-closing file handles that might corrupt the encryption keys or drive format. Should I use exFAT instead of NTFS?

What I've tried:

  1. Making sure the "Restore Point" feature is disabled for the drive.
  2. Making sure the drive was set for "Quick Removal".
  3. Re-formatted the drive with NTFS again and re-encrypted with Bitlocker.
  4. Trying another computer (same problem).

I have the issue since Windows 8 and after some trial & error I figured out, that the running Taskmanager (Taskmgr.exe) blocks the eject of external HDDs. My guess is, that it is blocked because of the new disk IO graphs. I've already reported it, but since you still get it in Windows 10, the issue is still not solved.