How does a Windows antivirus hook into the file access process?

The subject says it all. A normal antivirus has to intercept all file accesses, scan the files and then optionally deny access to the file (possibly even displaying a prompt to the user). How can this be done?

I'm aware of a method called API hooking, but that's a really dirty undocumented hack - and as such isn't really reliable. What's the "official" way of doing this?

Alternatively, I would be interested in intercepting the loading of executable modules (.DLL, .EXE, etc.), not just arbitrary file reads.

Solution 1:

In the recent versions of windows (at least XP onwards) there is the concept 'filters' which can be viewed using MS Filter Manager, (fltmc.exe from a command prompt)

This provides a low level I/O hook that AV programs can access and automatically register to be passed all I/O requests to the file system. It is a kit you can get the drivers for an develop your own filters for.

http://www.microsoft.com/whdc/driver/filterdrv/default.mspx is a starting place to get in depth info.

Solution 2:

As you already noted, hooking is a key to what of-the-shelf AV software with "realtime" protection does.

You could have a look on the (widely discussed) winpooch, which already does API Hooking, but there are some major flaws in this software. Sourceforge of Winpooch

There is also an article on Codeproject on API hooking, providing some library to do hooking "in three layers". Dll Injection is somewhat hard, as you can image. CodeProject: EasyHook, reinvention of API Hooking

As you are probably interested in Antivirus strategies, i also suggest having a look at ClamAV, or WinClam, which is opensource (under GPL) ClamAV for windows

But i do not have a clue how to do API hooking with C#, i have to admit. In C / C++ this is (quite) easy...

ADD ON You may be interested in the sources of FileMon, a widely known FileSystem Monitor that was once by SysInternals and now by Microsoft: It uses Driver-Filter API by Microsoft, which is at least known as fragile.

Link may be found here in Sysinternals forum

Solution 3:

Through File System Filter Drivers. However, implementing such drivers is quite complicated and "fragile".

Solution 4:

File access is monitored using filesystem filter driver, which works in kernel mode. Filter drivers can be not just notified about filesystem operations, but alter the data passed via filters or deny filesystem requests.

You can create a minifilter yourself, yet maintenance and support of your kernel-mode code can be non-trivial, especially without kernel-mode development experience. One of problems is conflicts between various filters.

Our company offers CallbackFilter product, which provides a ready-to-use driver and lets you write business logic, related to filtering, in user mode.