Array placement-new requires unspecified overhead in the buffer?

5.3.4 [expr.new] of the C++11 Feb draft gives the example:

new(2,f) T[5] results in a call of operator new[](sizeof(T)*5+y,2,f).

Here, x and y are non-negative unspecified values representing array allocation overhead; the result of the new-expression will be offset by this amount from the value returned by operator new[]. This overhead may be applied in all array new-expressions, including those referencing the library function operator new[](std::size_t, void*) and other placement allocation functions. The amount of overhead may vary from one invocation of new to another. —end example ]

Now take the following example code:

void* buffer = malloc(sizeof(std::string) * 10);
std::string* p = ::new (buffer) std::string[10];

According to the above quote, the second line new (buffer) std::string[10] will internally call operator new[](sizeof(std::string) * 10 + y, buffer) (before constructing the individual std::string objects). The problem is that if y > 0, the pre-allocated buffer will be too small!

So how do I know how much memory to pre-allocate when using array placement-new?

void* buffer = malloc(sizeof(std::string) * 10 + how_much_additional_space);
std::string* p = ::new (buffer) std::string[10];

Or does the standard somewhere guarantee that y == 0 in this case? Again, the quote says:

This overhead may be applied in all array new-expressions, including those referencing the library function operator new[](std::size_t, void*) and other placement allocation functions.

Solution 1:

Update

Nicol Bolas correctly points out in the comments below that this has been fixed such that the overhead is always zero for operator new[](std::size_t, void* p).

This fix was done as a defect report in November 2019, which makes it retroactive to all versions of C++.

Original Answer

Don't use operator new[](std::size_t, void* p) unless you know a-priori the answer to this question. The answer is an implementation detail and can change with compiler/platform. Though it is typically stable for any given platform. E.g. this is something specified by the Itanium ABI.

If you don't know the answer to this question, write your own placement array new that can check this at run time:

inline
void*
operator new[](std::size_t n, void* p, std::size_t limit)
{
    if (n <= limit)
        std::cout << "life is good\n";
    else
        throw std::bad_alloc();
    return p;
}

int main()
{
    alignas(std::string) char buffer[100];
    std::string* p = new(buffer, sizeof(buffer)) std::string[3];
}

By varying the array size and inspecting n in the example above, you can infer y for your platform. For my platform y is 1 word. The sizeof(word) varies depending on whether I'm compiling for a 32 bit or 64 bit architecture.

Solution 2:

Update: After some discussion, I understand that my answer no longer applies to the question. I'll leave it here, but a real answer is definitely still called for.

I'll be happy to support this question with some bounty if a good answer isn't found soon.

I'll restate the question here as far as I understand it, hoping that a shorter version might help others understand what's being asked. The question is:

Is the following construction always correct? Is arr == addr at the end?

void * addr = std::malloc(N * sizeof(T));
T * arr = ::new (addr) T[N];                // #1

We know from the standard that #1 causes the call ::operator new[](???, addr), where ??? is an unspecified number no smaller than N * sizeof(T), and we also know that that call only returns addr and has no other effects. We also know that arr is offset from addr correspondingly. What we do not know is whether the memory pointed to by addr is sufficiently large, or how we would know how much memory to allocate.

You seem to confuse a few things:

1. Your example calls operator new[](), not operator new().

2. The allocation functions do not construct anything. They allocate.

What happens is that the expression T * p = new T[10]; causes:

1. a call to operator new[]() with size argument 10 * sizeof(T) + x,

2. ten calls to the default constructor of T, effectively ::new (p + i) T().

The only peculiarity is that the array-new expression asks for more memory than what is used by the array data itself. You don't see any of this and cannot make use of this information in any way other than by silent acceptance.

If you are curious how much memory was actually allocated, you can simply replace the array allocation functions operator new[] and operator delete[] and make it print out the actual size.

Update: As a random piece of information, you should note that the global placement-new functions are required to be no-ops. That is, when you construct an object or array in-place like so:

T * p = ::new (buf1) T;
T * arr = ::new (buf10) T[10];

Then the corresponding calls to ::operator new(std::size_t, void*) and ::operator new[](std::size_t, void*) do nothing but return their second argument. However, you do not know what buf10 is supposed to point to: It needs to point to 10 * sizeof(T) + y bytes of memory, but you cannot know y.