C# DllImport with C++ boolean function not returning correctly

I found the solution for your problem. Your declaration should be preceded with this marshaling: [return:MarshalAs(UnmanagedType.I1)]

so everything should look like this:

[DllImport("Whisper.dll", EntryPoint="Exist", CallingConvention=CallingConvention.Cdecl)]  
[return:MarshalAs(UnmanagedType.I1)]  
public static extern bool Exist([MarshalAs(UnmanagedType.LPStr)] string name);

I tested it in my very simple example and it worked!

EDIT
Why this happens? C defines bool as 4 bytes int (as some of you have said) and C++ defines it as 1 byte. C# team decided to use 4 byte bool as default during PInvoke because most of the system API function use 4 bytes values as bool. If you want to change this behavior you have to do it with marshaling specifying that you want to use 1 byte value.


C'sbool is actually int, as there is no boolean type in the original C language. That means that if C#'s DLLImport is designed to interop with C code, then they will expect that C#'s bool to correspond to C's int. While this still doesn't explain why false would become true, fixing it should fix the problem.

http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.unmanagedtype.aspx

This says that UnmanagedType.Bool is the Win32 BOOL, which is an int.


This is actually caused by EAX not being fully cleared out by typical C++ code that returns a bool. It's typical for EAX to contain some bogus value when entering a function, and to return false the compiler would typically emit xor al, al. This clears out only the LSB of EAX, and causes C# code to interpret the resulting non-zero value as true instead of false.


Perhaps marshaling the argument of the function might help:

[MarshalAs(UnmanagedType.LPStr)]

Here is how the declaration should look like:

[DllImport("Whisper.dll", EntryPoint="Exist", CallingConvention=CallingConvention.Cdecl)]
        public static extern bool Exist([MarshalAs(UnmanagedType.LPStr)] string name);