Numerous instances of VBCSCompiler.exe
Solution 1:
Hmm, there is no obvious repro scenario and nobody else complains about this. Your solution is not unusual at all. Pegging the cpu to 100% and getting the VBCSCompiler process to swallow ~1.5 GB isn't very hard on a large project but it is squeaky clean when I look at mine.
First possible failure scenario that you have some uninstalled beta bits lying around, a very common problem. Use the debugger to have a look-see. Use Debug > Attach to Process and pick one of the running instances. Then Debug > Break All and Debug > View > Modules. Pay attention to the version numbers and timestamps, they ought to look like this:
Note that intentionally hid some columns to keep it readable. Time stamps are CST timezone.
That's the server side. The client side that had the bug you discovered is located in C:\Program Files (x86)\MSBuild\14.0\Bin\Microsoft.Build.Tasks.CodeAnalysis.dll. Have a look at its properties, mine is 85,192 bytes and created on Sunday, June 21, 2015, 7:06:54 PM, File version number 1.0.0.50618. You can look at the file with a decompiler like Reflector or ILSpy, navigate to BuildClient.TryAllProcesses(). The relevant line with the bug fix is:
for (int i = 1; File.Exists(string.Format(@"\\.\pipe\{0}", pipeName)); i++)
The buggy version was missing \\.\pipe\.
Note how error checking is very inadequate in the above snippet, File.Exists() returns false for many reasons. Also the basic reason the bug wasn't discovered earlier. That enables several possible failure modes, the kind enabled if your machine is infected by the typical shrinked-wrapped malware that programmers voluntarily install. The server and client code connect through each other through a named pipe with a special name. Something you can see in Task Manager, Processes tab. Use View > Select Columns (Win8 and up: right-click a column header) and tick the "Command line" option:
Note the -pipename argument. If the File.Exists() call returns false then MSBuild will start VBCSCompiler.exe again. If you see all these instances running with the same -pipename argument then you have software running on your machine that is interfering with normal named pipe usage. First thing you'd consider then is to look for a less aggressive anti-malware solution. You can write a little test program that uses the System.IO.Pipes namespace to get a better exception message.
Solution 2:
Any idea why there are so many of these tasks running?
Roslyn uses a shared compiler process that keeps your compiled code in memory for reuse in subsequent compiles. So the second compile will be quicker, but as you've seen there's a memory overhead.
Any way to stop this from happening?
Yes. From here there's a property of the compile task in msbuild that turns off the shared compiler, and it's set to true by default.
So in each project you'll have to add this property to the project file. Or in Visual Studio 2015 there's now shared projects, where you could add this property to the shared project and then include that shared project in all the other projects that need this setting.
<PropertyGroup>
<UseSharedCompilation>false</UseSharedCompilation>
</PropertyGroup>
Solution 3:
As of November 22, 2015 this issue was still happening to me in the Visual Studio 2015 community edition. My laptop was starting to double as a space heater with all of the instances of VBCSCompiler running full throttle.
The only fix that worked for me was to locate the file VBCSCompiler.exe in the /bin/roslyn directory of the web application and change security permissions on it.
You need to deny read & execute permission for the AppPool that your web app is running under.
Solution 4:
According to Slaks, you can disable roslyn (which is what the VBCSCompiler.exe appears to be) by setting the DisableRoslyn environment variable to true.
See http://blog.slaks.net/2014-05-21/exploring-roslyn-part-2-inside-end-user-preview/ for more information.
Note, the link above is for the preview, but I can't imagine that it would have changed much between then and now.