How to download TMF files for tracefmt from Microsoft Symbol Server

The Windows Event Tracing framework (ETW) can be used to log a lot of information about the system internals. The tracefmt.exe tool that comes with Windows SDK can be used to convert the .etl log files into text.

However, it requires .tmf message format files to process the messages. Otherwise the data fill just appear as unknowns:

Unknown( 25): GUID=72e5b5cd-5b46-3568-7f3a-3eb074bedc0f (No Format Information found).

How can I download the needed .tmf files?

Solution 1:

1. Identify which .sys file is related to the GUID.

Sometimes simply searching the GUID online will identify the related .sys file. If it does not, one can do a binary search among the .sys files in C:\Windows\System32.

Any binary grep tool can be used, but the GUID bytes have to be reordered to little-endian order. For example 72e5b5cd will appear as 0xcd 0xb5 0xe5 0x72. I have used the following Python one-liner to perform the search:

python -c 'import sys,uuid; print([x for x in sys.argv[1:] if uuid.UUID("72e5b5cd-5b46-3568-7f3a-3eb074bedc0f").bytes_le in open(x,"rb").read()])' *.sys

That particular GUID will appear in winusb.sys.

Note that only 64-bit applications can access the real system files, 32-bit applications will not see them.

2. Download the .pdb file for the .sys

For this step you'll need symchk.exe from Windows SDK. Create a new directory to store the files, here I've used C:\tracing:

symchk.exe C:\Windows\System32\drivers\winusb.sys /s "SRV*C:\tracing*http://msdl.microsoft.com/download/symbols" /od

3. Extract the trace formats from the .pdb

For this step you'll need tracepdb.exe from Windows SDK. I've placed the output in the same directory as the previous step.

tracepdb.exe -f C:\tracing\winusb.pdb\450C4140E6D427E5ED2F1820DE4C7A2E1\winusb.pdb -p C:\tracing

Note that the hex string will depend on the system file version. Check the file listing to see under what name the symbols have been downloaded.

Now you will see a listing of TMF files being generated:

....
tracepdb: info BNP0000: WPPFMT generating C:\tracing\72e5b5cd-5b46-3568-7f3a-3eb074bedc0f.tmf for C:\tracing\winusb.pdb\450C4140E6D427E5ED2F1820DE4C7A2E1\winusb.pdb
...

Hopefully you'll see the GUID you want in that listing. If not, it either wasn't actually in that file, or it has been removed from the public symbols for some reason.

Note that if you are debugging an .etl collected from another computer, you'll need to collect the .sys files from that computer also to get the correct symbol versions.

4. Decode the trace

Now that we have the .tmf files, they can be passed to tracefmt.exe:

tracefmt.exe mylog.etl -p C:\tracing -o mylog.txt

The command will show result count as:

Processing completed   Buffers: 156, Events: 274074, EventsLost: 0 :: Format Errors: 0, Unknowns: 8409

In this case most of the events were decoded ok, but there are still a few unknowns. Check out which lines still have Unknown in them, and repeat from step 1.