Retrieving files from directory that contains large amount of files
I have directory that contains nearly 14,000,000 audio samples in *.wav format.
All plain storage, no subdirectories.
I want to loop through the files, but when I use DirectoryInfo.GetFiles()
on that folder the whole application freezes for minutes!
Can this be done another way? Perhaps read 1000, process them, then take next 1000 and so on?
Have you tried EnumerateFiles method of DirectoryInfo class?
As MSDN Says
The
EnumerateFiles
andGetFiles
methods differ as follows: When you useEnumerateFiles
, you can start enumerating the collection ofFileInfo
objects before the whole collection is returned; when you useGetFiles
, you must wait for the whole array ofFileInfo
objects to be returned before you can access the array. Therefore, when you are working with many files and directories,EnumerateFiles
can be more efficient.
In .NET 4.0, Directory.EnumerateFiles(...)
is IEnumerable<string>
(rather than the string[]
of Directory.GetFiles(...)
), so it can stream entries rather than buffer them all; i.e.
foreach(var file in Directory.EnumerateFiles(path)) {
// ...
}