Visual Studio 2010 & 2008 can't handle source files with identical names in different folders?

Direct Question: If I have two files with the same name (but in different directories), it appears that only Visual Studio 2005 can handle this transparently?? VS 2008 & 2010 require a bunch of tweaking? Aside from my naming convention, am I doing something wrong?

Background:

I'm developing C++ statistical libraries... I have two folders:

/ Univariate

Normal.cpp
Normal.h
Beta.cpp
Beta.h
Adaptive.cpp
Adaptive.h

/ Multivariate

Normal.cpp
Normal.h
Beta.cpp
Beta.h
Adaptive.cpp
Adaptive.h

I need to support cross compilation -- I'm using g++/make to compile these same files into a library in Linux. They work just fine.

I had been using Visual Studio 2005 without issue, but I need to upgrade to Visual Studio 2008 or 2010 (currently drooling over nVidia's nsight tool). However, I'm having trouble if I add files to a project with the same name (even if they're in a different directory). I'm willing to change my naming convention, but I'm curious if others have encountered this problem and have found any well documented solutions??

I'm further boggled by the fact that if I upgrade from 2005 projects to 2010 projects, it appears that VS 2010 is able to correctly handle two source files with the same name in different directories; however, if I remove one of the duplicate files and then add it back to the project I am greeted by the following warning:

Distributions\Release\Adaptive.obj : warning LNK4042: object specified more than once; extras ignored

Now I have the intermediate directory specified as $(ProjectName)\$(Configuration) -- I need to have my object files in a different location from my source tree. So I can see why it's copying the object files on top of each other, but when the projects are converted from 2005 to 2008 or 2010, a bunch of conditional compiles are added:

<ObjectFileName Condition="'$(Configuration)|$(Platform)'=='Release|x64'">$(IntDir)%(Filename)1.obj</ObjectFileName>
<XMLDocumentationFileName Condition="'$(Configuration)|$(Platform)'=='Release|x64'">$(IntDir)%(Filename)1.xdc</XMLDocumentationFileName>

These are accessible from the Source file Properties page in C/C++ -> Output Files -> "Object File Name" & "XML Documentation File Name". But if I simply add the file directly (or remove and re-add them), VS doesn't complain until I try to compile, but also never adds the conditional directives -- So in order for things to work correctly, I have to add the conditional directives myself for every single configuration. Am I making a mistake / poor assumption or have I uncovered a valid bug in VS 2008 / 2010?

So @Hans Passant pointed in the right direction, Thanks!! You don't have to list the file, a folder is sufficient. Then if you look in the defined macros at the bottom of the VS 2010 list, you'll see:

%(RelativeDir)/ Univariate/

The problem, as posted, was actually a simplified version of what I'm working on -- a couple of levels of folders in a single project and there are a couple of name conflicts. Hence, I really wanted someway to just "fix" it...

If you right click on the project in the solution explorer, choose C/C++ -> "Output Files" and type the following into the "Object File Name" box:

$(IntDir)/%(RelativeDir)/

Note that I also selected (All Configurations, All Platforms) from the drop downs. This will compile every file in a directory hierarchy which mirrors the source tree. VS2010 will begin the build by creating these directories if they don't exist. Further, for those who hate white space in their directory names, this macro does remove all spaces, so there is no need to play around with double quotes when using it.

This is exactly what I wanted -- identical to the way my Makefiles work on the Ubuntu side, while still keeping the source tree clean.

This is easy to fix in the IDE. Click the first file in the folder, Shift+Click the last file so all of them are selected. Right-click, Properties, C++, Output Files. Change the Object File Name from $(IntDir)\ to, say, $(IntDir)\Univariate\. You can repeat for the Multivariate file group although that's not strictly necessary.

You're right, VS can't handle that, and never could. The root problem is that it generates a .obj file for each .cpp file in the project, and they're all placed in the same folder. So you end up with multiple .cpp files compiling to Adaptive.obj in your case, for example.

At least the linker generates a warning for it now. That wasn't always the case.

You should be able to work around this by ensuring the files use different Intermediate Directory paths, but it is a bit of a hack around something that ought to be possible.

Of course, you could always file a bug report or feature request on it on Microsoft Connect

Note that in my instance (Visual Studio 2013 with VS2010 platform toolset), using $(IntDir)\%(RelativeDir) doesn't work correctly, and it ignores the intermediate directory which results in linker errors when building multiple configurations because all of the object files for every configuration (ie, Debug and Release) are placed in the same folder. These go away if you clean the project when switching configurations.

Error Example:

MSVCRTD.lib(MSVCR100D.dll) : error LNK2005: _fclose already defined in LIBCMTD.lib(fclose.obj)

Solution:

$(IntDir)\%(Directory)

To get around this, I had to use $(IntDir)\%(Directory) which correctly placed all of the *.obj files under the intermediate directory and allowed building and linking multiple configurations without cleaning. The only downside is the entire (potentially long) folder hierarchy that your files are in will be completely re-created under the Debug/Release/etc folder.

The other solutions in this thread suffer from the RelativeDir-based ../.. problems and having to set things manually on each source file.

Not to mention, they wreck /MP. Any solution which specifies an exact .obj for %(ObjectFileName) will result in a different /Fo for each .cpp file (to map it to a specific .obj file) passed to CL.exe and thus Visual Studio can't batch them. Without batching several .cpp files with identical commandlines (including /Fo) the /MP can't work.

Here's a new approach. This works on vs2010 through to vs2015 at least. Add this to your vcxproj in the <project>

<!-- ================ UNDUPOBJ ================ -->
<!-- relevant topics -->
<!-- https://stackoverflow.com/questions/3729515/visual-studio-2010-2008-cant-handle-source-files-with-identical-names-in-diff/26935613 -->
<!-- https://stackoverflow.com/questions/7033855/msvc10-mp-builds-not-multicore-across-folders-in-a-project -->
<!-- https://stackoverflow.com/questions/18304911/how-can-one-modify-an-itemdefinitiongroup-from-an-msbuild-target -->
<!-- other maybe related info -->
<!-- https://stackoverflow.com/questions/841913/modify-msbuild-itemgroup-metadata -->
<UsingTask TaskName="UNDUPOBJ_TASK" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
  <ParameterGroup>
    <OutputDir ParameterType="System.String" Required="true" />
    <ItemList ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
    <OutputItemList ParameterType="Microsoft.Build.Framework.ITaskItem[]" Output="true" />
  </ParameterGroup>
  <Task>
    <Code><![CDATA[
            //general outline: for each item (in ClCompile) assign it to a subdirectory of $(IntDir) by allocating subdirectories 0,1,2, etc., as needed to prevent duplicate filenames from clobbering each other
            //this minimizes the number of batches that need to be run, since each subdirectory will necessarily be in a distinct batch due to /Fo specifying that output subdirectory

            var assignmentMap = new Dictionary<string,int>();
            HashSet<string> neededDirectories = new HashSet<string>();
            foreach( var item in ItemList )
            {
              //solve bug e.g. Checkbox.cpp vs CheckBox.cpp
              var filename = item.GetMetadata("Filename").ToUpperInvariant(); 

              //assign reused filenames to increasing numbers
              //assign previously unused filenames to 0
              int assignment = 0;
              if(assignmentMap.TryGetValue(filename, out assignment))
                assignmentMap[filename] = ++assignment;
              else
                assignmentMap[filename] = 0;

              var thisFileOutdir = Path.Combine(OutputDir,assignment.ToString()) + "/"; //take care it ends in / so /Fo knows it's a directory and not a filename
              item.SetMetadata( "ObjectFileName", thisFileOutdir );
            }

            foreach(var needed in neededDirectories)
              System.IO.Directory.CreateDirectory(needed);

            OutputItemList = ItemList;
            ItemList = new Microsoft.Build.Framework.ITaskItem[0];

        ]]></Code>
  </Task>
</UsingTask>

<Target Name="UNDUPOBJ">
  <!-- see stackoverflow topics for discussion on why we need to do some loopy copying stuff here -->
  <ItemGroup>
    <ClCompileCopy Include="@(ClCompile)"/>
    <ClCompile Remove="@(ClCompile)"/>
  </ItemGroup>
  <UNDUPOBJ_TASK OutputDir="$(IntDir)" ItemList="@(ClCompileCopy)" OutputItemList="@(ClCompile)">
    <Output ItemName="ClCompile" TaskParameter="OutputItemList"/>
  </UNDUPOBJ_TASK>
</Target>
<!-- ================ UNDUPOBJ ================ -->

And then modify <project> so that it reads:

<Project InitialTargets="UNDUPOBJ" ...

The result will be something like myproj/src/a/x.cpp and myproj/src/b/x.cpp compiling to Debug/0/x.obj and Debug/1/x.obj. RelativeDirs arent employed and so aren't a problem.

Additionally, in this case, there will be only two different /Fo passed to CL.exe: Debug/0/ and Debug/1/. Consequently, no more than two batches will get issued to CL.exe, allowing the /MP to work more efficiently.

Other approaches would be basing the .obj subdirectories on the .cpp subdirectories, or making the .obj filename contain some memento of the original .cpp directory so that you can readily see a .cpp->.obj mapping, but those result in more /Fo and therefore less batching. Future work could dump a mapping file for quick reference, perhaps.

See this for more details on /MP and batching : MSVC10 /MP builds not multicore across folders in a project

I've tested this in production for quite a while on vs2010 and vs2015 on a variety of toolchains. It seems bulletproof, but there's always a chance it may interact badly with other msbuild customizations or exotic toolchains.

Starting in vs2015, if you get a warning "warning MSB8027: Two or more files with the name of X.cpp will produce outputs to the same location" then you can add this to your project or msbuild files:

<PropertyGroup Label="Globals"><IgnoreWarnCompileDuplicatedFilename>true</IgnoreWarnCompileDuplicatedFilename></PropertyGroup>

See more at https://connect.microsoft.com/VisualStudio/feedback/details/797460/incorrect-warning-msb8027-reported-for-files-excluded-from-build and How to suppress specific MSBuild warning

UPDATE SEP 2019:

I believe there is a problem with the above where the ItemList can be arbitrarily ordered resulting in reassignments of a given source file to varying numbered directories. Thus whenever the source file list changes, duplicate copies of the object may appear. This may also cause dependency tracking to get a bit messed up, since the input->output mapping can get changed surprisingly so that the timestamps make no sense.

We could fix this by hashing a sorted ItemList and prefixing that to the output directory, but this will create extra rebuilds and a lot of litter.

This could be refined by dropping a file listing input->output mappings, and having the mapping of a source file to destination numbered-directory be a one-time only event (until the project is cleaned).

I have yet to investigate this more deeply.

Visual Studio 2010 & 2008 can't handle source files with identical names in different folders?

Related

Recent Posts