How can I see which DLLs included in Windows contain icons?

Use IconsExtract from NirSoft.

It scans a folder you select and can list any icons and files which contain icons.

Big ones:

%windir%\system32\shell32.dll           Standard
%windir%\System32\wmploc.DLL            OK
%windir%\system32\setupapi.dll          OK, Hardware / Periphals
%windir%\system32\ddores.dll            OK, lot of devices
%windir%\System32\ieframe.dll           IE and warnings, mixed ugly
%windir%\system32\netshell.dll          Ugly and non ugly
%windir%\System32\imageres.dll          Ugly, New ones
%windir%\System32\pifmgr.dll            Ugly, Win95 Ones
%windir%\System32\moricons.dll          Ugly, very old
%windir%\System32\mmcndmgr.dll          Ugly, Very old
%windir%\System32\compstui.dll          Ugly, Very old

Small ones:

%windir%\system32\accessibilitycpl.dll
%windir%\explorer.exe
%windir%\system32\gameux.dll            Some gaming icons
%windir%\system32\mmres.dll             Audio
%windir%\system32\mstscax.dll           Warnings
%windir%\System32\netcenter.dll         Network
%windir%\System32\networkexplorer.dll   
%windir%\system32\networkmap.dll
%windir%\System32\pnidui.dll            Ugly Exlamations
%windir%\system32\SensorsCpl.dll        Handheld devices
%windir%\system32\xpsrchvw.exe 
%windir%\system32\UIHub.dll
%windir%\system32\vpc.exe
        - provided as a free download for Win7 owners
%windir%\system32\wmp.dll               Media player
%windir%\system32\wpdshext.dll          Battery and arrows
%windir%\system32\wucltux.dll

The classic additional icon repository of Win95 is %SystemRoot%\system32\moricons.dll. The file is still included with Windows 7.