Windows Service Limit Crashes Services on Startup
We have developed a custom Windows service in C# as part of a large Enterprise application. Our QA department tests multiple versions of this service.
The QA lab has several (over 20) copies of this service installed on one Windows 2003 test box. Each copy is in its own folder and has a unique service name, though each executable file is named the same (OurWindowsService.exe, for example). Each service uses the same Windows credentials (a domain user).
The purpose of this service is to handle MSMQ messages. The queued messages do all sorts of important stuff.
For some reason, they can run only 5 of these services at a time. When we start a 6th, the service crashes on startup.
For example, I can start #1, #2, #3, #4, and #5. When I start #6, it crashes. However, if I stop #1 and start #6, #6 runs fine, and now #1 fails to start.
When the services crash, the following error appears in the Windows event log:
Faulting application OurWindowsService.exe, version 5.40.1.1, faulting module kernel32.dll, version 5.2.3790.4480, fault address 0x0000bef7.
I was able to use WinDbg to generate a postmortem dump file. The dump file revealed that the crash occurs trying to delay load SHLWAPI.dll:
0:000> kb100
ChildEBP RetAddr Args to Child
0012ece4 79037966 c06d007e 00000000 00000001 KERNEL32!RaiseException+0x53
0012ed4c 790099ba 00000008 0012ed08 7c82860c mscoree!__delayLoadHelper2+0x139
0012ed98 790075b1 001550c8 0012edac 0012fb34 mscoree!_tailMerge_**SHLWAPI_dll**+0xd
0012edb0 79007623 001550c8 0012edf8 0012edf4 mscoree!XMLGetVersionWithSupported+0x22
0012ee00 790069a4 aa06f1b0 00000000 000001fe mscoree!RuntimeRequest::GetRuntimeVersion+0x56
0012f478 790077aa 00000001 7903fb4c 0012fb34 mscoree!RuntimeRequest::ComputeVersionString+0x5bd
0012f89c 79007802 00000001 0012f8b4 7903fb4c mscoree!RuntimeRequest::FindVersionedRuntime+0x11c
0012f8b8 79007b19 00000001 00000000 aa06fa6c mscoree!RuntimeRequest::RequestRuntimeDll+0x2c
0012ffa4 79007c02 00000001 0012ffbc 00000000 mscoree!GetInstallation+0x72
0012ffc0 77e6f23b 00000000 00000000 7ffdf000 mscoree!_CorExeMain+0x12
0012fff0 00000000 79007bf0 00000000 78746341 KERNEL32!BaseProcessStart+0x23
I believe the error code handed to Kernel32.RaiseException, c06d007e, means Module Not Found, but I'm not certain.
Does this sound familiar to anyone? Are we hitting some limit on the number of service instances on some file name? Does MSMQ dislike more than 5 listening services?
If your service is loading a copy of the message queuing driver (Mqac.sys) per instance, it's likely that you are running out of System View Space memory pool. Each instance of MSMQ uses 4MB of this pool and by default the pool size is only 16MB.
Whether this is actually happening will really depend on how (if at all) your service uses the MSMQ system, and whether it references it directly (loads its libraries) or uses some other method such as sockets to communicate with it.
If it turns out you are loading multiple instances of MSMQ you can mitigate your problem by increasing the system view space pool. This can be done by:
- Open the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management.
- Create a new DWORD value called SystemViewSize.
- Calculate and specify this value as follows:
- Use this formula: (16 + (the number of Message Queuing resources loaded x 4)).
- For example, the value for a cluster with three Message Queuing resources equals 28.
- Reboot