Getting console message: ipc_kmsg_copyout_header: can't grow user ipc space. Any Mac OS X kernel gurus in here?

Ever since upgrading from Mac OS X 10.6.5 to 10.6.7, my computer has started locking up frequently (usually at least once every couple of days). I will get the spinning pinwheel and the system will become non-responsive (both gui and ssh, etc). This state will last indefinitely, requiring a forced restart. It will enter this irrecoverable spin when I am actively using the computer or even in my absence, "idle" for hours or days. This is not a kernel panic.

Upon restarting, I check the console logs to see what might be wrong. In every single case, there is always the same message that appears right before the beginning of the system startup messages. It reads something like:

2011/6/06 9:41:51 AM kernel ipc_kmsg_copyout_header: can't grow user ipc space

with, of course, the date and time of the last instance that the computer was responsive. Nothing else unusual appears before this message. Just standard console stuff.

Googling this message, I only came across where this message appears in the source code ipc_kmsg.c, which appear to be components of the freebsd and mach kernels

Here are links to the relevant source:

1) http://fxr.watson.org/fxr/source/osfmk/ipc/ipc_kmsg.c?v=xnu-1456.1.26

    2963                 if (kr != KERN_SUCCESS) {
    2964                     /* space is unlocked */
    2965 
    2966                     if (kr == KERN_RESOURCE_SHORTAGE) {
    2967                         printf("ipc_kmsg_copyout_header: can't grow kernel ipc space\n");
    2968                         return (MACH_RCV_HEADER_ERROR|
    2969                             MACH_MSG_IPC_KERNEL);
    2970                     } else {
    2971                         printf("ipc_kmsg_copyout_header: can't grow user ipc space\n");
    2972                         return (MACH_RCV_HEADER_ERROR|
    2973                             MACH_MSG_IPC_SPACE);
    2974                     }
    2975                 }

2) http://fxr.watson.org/fxr/ident?v=xnu-1456.1.26;im=excerpts;i=MACH_MSG_IPC_SPACE

    658 #define MACH_MSG_IPC_SPACE              0x00002000
    659                 /* No room in IPC name space for another capability name. */

3) (can't post 3rd link. new user -_-)

    720 #define MACH_RCV_HEADER_ERROR           0x1000400b
    721                 /* Error receiving message header.  See special bits. */

I don't want to pretend to know exactly what's going on here, but it looks like the kernel is running out of open ports for ipc? If this is the case, what could be causing this problem? Shouldn't the system free up ports that aren't in use? I can't think of anything I have installed that could use up all the ipc ports.

I haven't seen any other posts anywhere on the internets about others having this problem, but I can't imagine I'm the only person to have it.

Thanks, any help would be appreciated.

This is a shot in the dark, but are you using Mozy for backup? I helped another user with a problem where his kernel was running out of IPC-related resources (specifically Mach ports in his case), and he ended up tracking it down to Mozy.

See Some Mac applications crash frequently, with "__THE_SYSTEM_HAS_NO_PORT_SETS_AVAILABLE__" in backtrace

Even if you're not using Mozy, you might try what he did and turn on the "Ports" column in Activity Monitor and see who's using up Mach ports. Obviously you'll have to do this before the problem happens, maybe leaving it open so you can monitor things for early signs that things are about to fail. You might also turn on the "Messages Sent" and "Messages Received" columns to see which process are sending and receiving the most IPC messages. But before you get surprised at all the messages to/from the kernel_task, be sure to compare to a machine that's working normally, that was rebooted at about the same time, to get a baseline. You can also look at the list of processes in Activity Monitor to see if there are excessive copies of a given binary running.

Since you seem willing to dig into kernel sources, you might also want to read up on how to debug kernel problems:

• Apple's Mac OS X Kernel Programming Guide
• Technical Note TN2063: Understanding and Debugging Kernel Panics
• Technical Note TN2118: Kernel Core Dumps

After reading up on kernel debugging, you might try setting sudo nvram boot-args="debug=0x144" to enable several popular kernel debugging options, and then, the next time the problem hits, you can press your power key to force a kernel panic so you can attach with GDB from another machine and poke around. If you want your power key to go back to normal operation, use sudo nvram boot-args="debug=0x140" (leave the other cool options, on, but disable the power key hack) or sudo nvram -d boot-args (remove then entire "boot-args" NVRAM variable).