How can I use Web.debug.config in the built-in visual studio debugger server?

This is a known bug. That feature can be used right now only as part of the deploy process.

https://connect.microsoft.com/VisualStudio/feedback/details/523221/have-web-debug-config-apply-during-development

Please upvote it, if you encounter this too, so this will be fixed ASAP.

This is actually quite simple to do and, believe it or not, it seems this is the way VS is designed to work.

Add the following lines verbatim right before the closing "Project" tag of the .csproj file of the project that contains web.config.

<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Web\Microsoft.Web.Publishing.Tasks.dll" />
<Target Name="Transform">
    <MakeDir Directories="obj\$(Configuration)" Condition="!Exists('obj\$(Configuration)')" />
    <TransformXml Source="Web.Config" Transform="Web.$(Configuration).config" Destination="obj\$(Configuration)\Web.config" StackTrace="true" />
</Target>

Put the following lines verbatim to the post-build event in the project properties of the project that contains the web.config file. Do this for each build configuration you want the transformations to run for.

"$(MSBUILDBINPATH)\msbuild" "$(ProjectPath)" /t:Transform /p:Configuration=$(ConfigurationName);Platform=AnyCPU
xcopy "$(ProjectDir)obj\$(ConfigurationName)\Web.Config" "$(ProjectDir)". /F /R /Y

I had solved this in a simpler way, by adding this at the end of the .csproj file, right before the tag. This is similar to keitn's answer, with the difference that it doesn't use a post build event.

<Target Name="BeforeBuild">
    <TransformXml Source="Web.config" Transform="Web.$(Configuration).config" Destination="Web.config" />
</Target>