How to prevent Windows Explorer from slowly reading file content to create metadata?
We have directories containing hundreds of video files. When using Windows Explorer (Windows 7 64 Pro) to browse and rearrange these directories, Explorer slows down to an unusable level, uses gigabytes of RAM, and pre-empts or slows other programs (such as Media Center) from using the same disk while this is in progress.
This is accompanied by a very slow-growing green bar in the Address slot.
What Explorer appears to be doing is reading the video file content to produce thumbnails, or obtain other metadata. This is not useful for our purposes, so we'd like to disable this behavior. How does one do that?
These threads may be related:
Superuser Prevent Windows Explorer from trying to extract metadata. [Explanation added 2014-04-11] This solution disables the registry's listing for a PropertyHandler for a particular filename extension. This could be useful, but is a little drastic -- would disable display of properties in Explorer for all directories, not just problem ones. (See more discussion in the answers.)
Superuser: "Sorting by Date - Very Slow" : The 'Date' column attempts to get EXIF data, and is distinct from the filesystem 'Date created' and 'Date modified' columns.
Microsoft: "Explorer - sort by date very very slow"
2014-04-14: Summary of suggestions and findings
I think at this point we're well satisfied with the answers, and remaining symptoms are unrelated to Windows Explorer. So, herewith a summary of suggestions and a few other things we learned.
General considerations
Disk wake-up: Initial slowness when accessing a disk after a long period of disuse may be attributable to the disk having gone to sleep.
Disk errors: Obviously if there are hardware errors which compel retries, this will gravely slow down Explorer and any other program.
Disk contention: Other programs performing intensive activity on the same drive will obviously contend with, and slow, Explorer's attempts to display a drive's contents. If Explorer needs to read only the directory, then impact may be minimal, but if Explorer also needs to read each file for metadata, then contention problems will be greatly magnified.
Disk contention from anti-virus software: Explorer's attempts to read metadata may prompt AV software to read and OK each file first. Consider disabling AV for trusted file types (for example, wtv files saved by Media Center).
Explorer settings
Disable display of thumbnails: "Organize" → "folder and search options" → "view" → put a checkmark in "always show icon instead of thumbnails" → press apply and close.
Disable gathering of video-specific metadata: Culprit folder → "properties" → "customize" → under "optimize this folder for:" open the drop down menu and choose "general items" → place the checkmark in the box under the drop down menu at "apply to all subfolders" → press apply and close.
Disable even more the gathering of video-specific metadata: In a culprit folder's "detail view", remove any columns based on video file metadata not in the file system's directory information. This includes the Length (duration) column, and also the Date column (based on video metadata, and distinct from the file system's "Date created" and "Date modified" columns.)
In the registry
Disable "Property Handlers" for video filename extensions (such as wtv, mp3 etc). This, of course, would apply system wide, not just for particular directories. Can be accomplished with RegEdit, or with 3rd party program ShellExView. (See Answers for details.)
More on thumbnail and other caches
We noted that after disabling thumbnails and metadata, when we re-enabled them, performance was speedy. (Ie: The green bar of slowness didn't return.) This suggests that the changes did something regarding the thumbnail cache(s), perhaps deleted them and later rebuilt them. Turns out that by default Explorer saves thumbnails in each directory (in thumbs.db) and also in the user's AppData\Local\Microsoft\Windows\Explorer directory in a set of thumbcache*.db files. It's possible these get into a bloated or otherwise slow state, and deleting them might prompt faster operation.
Not known (by us) is whether Explorer caches any other metadata info (such as Length, Date, Location, Protection and so on).
Other interactions
Not attributable to Explorer per se, but potentially useful clues.
We noted in Task Manager > Resource Monitor that Media Center was performing bad behavior that created a lot of disk activity: A media center component, ehrec.exe, would endlessly cycle through reading a few k of data from a particular set of video files -- on one drive, about 200 video files. It is as though Media Center is unable to complete reading metadata from those files, and just keeps trying to read files for which it doesn't have complete metadata. These files seem undamaged as far as the file system is concerned (can copy without problems). When we asked Explorer to display directories on this particular disk, conceivably Media Center's activity competed for disk access time. We disconnected this drive for other tests.
Thanks to all respondents.
Solution 1:
FINAL UPDATE:
OP has summarised all his findings completely and concisely in his question post. I see no reason to delete my suggestions, but i recommend you read his post instead of mine if you are for a quick solution to your problem.
You should try two things:
In Explorer, click on "organize" -> "folder and search options" -> "view" -> put a checkmark in "always show icon instead of thumbnails" -> press apply and close.
Now right-click on the culprit folder -> "properties" -> "customize" -> under "optimize this folder for:" open the drop down menu and choose "general items" -> place the checkmark in the box under the drop down menu at "apply to all subfolders" -> press apply and close. This will apply the new folder view settings to the chosen folder and all the subfolders which are contained within.
I am german and thus have a german version of windows, so maybe some of those options above are translated slightly differently, but you should be able to find them nonetheless.
Update 1:
I think you are on the right track with the metadata. Depending how the videos are encoded, the metadata could be either at the beginning, at the end, or even somewhere in the middle (which is rare, though). I am guessing that those video files were encoded with some unusual properties (you probably produced them yourself?) which makes explorer read in the whole file from beginning to the end to extract the metadata, which obviously takes a while if there are many big files in the folder. I have seen explorer read the full length of a giant exe file to display the embedded icon at the end.
So, i think you have it figured out there, identifying and disabling the columns which need to extract metadata from the view in explorer (together with the disabled thumbnails) should obviate explorer's need to read in those files, which should solve your problem.
Columns you should probably not use are something like: date taken (as mentioned in one of the links you posted, date taken is very differently from the creation date of the file), length, resolution, location.
Columns you should be safe to use would be attributes which can be read directly from the directory of the filesystem like: file creation date, file modified date, size, file type.
If you indeed need for your sorting some of the attributes which should be disabled, i think that maybe the most practical solution would be to look for an alternative file browser, and check if it handles the situation better. You could then use explorer like you are used to normally, and use the alternative file browser to handle your video folders.
You also have the possibility to perform many basic file-oriented operations from your built-in command line interpreter cmd, it doesn't care about metadata and can be a simple and efficient tool for copying, moving or deleting files and folders. You could then even automate things by using batch files. This is most probably not the solution you search for, though, since cmd doesn't even have a graphical user interface.
Update 2:
I just read your second update and i am happy to read that your problem seems resolved (for now, at least). Maybe it really was just a matter of the thumbnail cache getting overcrowded. I could imagine those thumbs.db files getting bigger and bigger, if you move files from folder to folder frequently. I suspect it actually keeps a thumbnail in that cache file for every file which ever was in the folder. Maybe there is some sort of garbage collection mechanism for those files, too, but it failed in your case.
So, if you were moving video files from folder to folder en masse, and always using the same folders for that (eg. not creating new ones) maybe we have found the source of your problem...
If your system shows the same symptoms again in the future, you could just try deleting the thumbnail cache. For that you need to:
"windowskey + r" -> input "cleanmgr" and press return -> choose the drive where the video files are on (only if you actually have multiple drives/partitions) -> choose "clear thumbnail cache" or something like that -> run cleaner
Solution 2:
There are three GPO settings when changed, will probably achieve the behavior that you want globally on the computer.
To get there, launch Group Policy Editor from a command-line.
gpedit.msc
Navigate to the following node.
User Configuration\Administrative Templates\Windows Explorer
The in-built documentation would be a good start to know what each setting does.
Do take note you need administrative rights to make changes to the GPO settings.
Solution 3:
My solution was arrived at indirectly. I noticed that MKV files contained a "date" that couldn't be changed by normal means so that the file would reflect when I got it, as shown in the file properties. In desperation I changed the column in windows file explorer from just "date" to "date created" and then set the view option to make this the default for all folders. Not only did this fix my sorting problem but the GROD was also cured. Not having to dig very deeply into the tagging structure of every file is obviously the reason for this cure.
Solution 4:
This answer is based on observations handling audio files on Windows 10, still it's worth a try.
I figured that when I named any of the parent folders containing audio files "music"[*], Windows would automatically infer that the folder view in explorer needed to be optimized for "music" and thus would read and display metadata. This happened although I had customized all folders for "General items" (in folder Properties -> Customize tab). Analyzing metadata takes relatively long. I did not want to wait, so I changed the name to "wav" and again customized the folders for "General items". Voila, access is lightning fast.
Consequently for video files, if you don't want metadata to be analyzed, my suggestion is:
Avoid the name "video" for any of the folders of your directory hierarchy where your video files reside in.
Otherwise, rename it and apply customization to it (and its subfolders) for "General items".
[*] Others have found that the name "audio" would trigger this mechanism.