NSI service on Win2008R2 using huge amount of RAM over time
You are probably running into this: http://answers.microsoft.com/en-us/windows/forum/windows_7-system/nsi-memory-leaks-fixed-yet/f3c3e9db-36a7-45df-9dbd-de9455378498
a memory leak in Windows 7 / Windows Server 2008 R2, introduced when you installed IE10 / IE11.
Workaround until they really fix it: uninstall IE10 and IE11 (and double check afterwards you really have IE9 or IE8). As of now (patched up to 10 July 2014) the issue isn't fixed.