"Proper" way to store binary data with C++/STL

In general, what is the best way of storing binary data in C++? The options, as far as I can tell, pretty much boil down to using strings or vector<char>s. (I'll omit the possibility of char*s and malloc()s since I'm referring specifically to C++).

Usually I just use a string, however I'm not sure if there are overheads I'm missing, or conversions that STL does internally that could mess with the sanity of binary data. Does anyone have any pointers (har) on this? Suggestions or preferences one way or another?

Solution 1:

vector of char is nice because the memory is contiguious. Therefore you can use it with a lot of C API's such as berkley sockets or file APIs. You can do the following, for example:

  std::vector<char> vect;
  ...
  send(sock, &vect[0], vect.size());

and it will work fine.

You can essentially treat it just like any other dynamically allocated char buffer. You can scan up and down looking for magic numbers or patters. You can parse it partially in place. For receiving from a socket you can very easily resize it to append more data.

The downside is resizing is not terribly efficient (resize or preallocate prudently) and deletion from the front of the array will also be very ineficient. If you need to, say, pop just one or two chars at a time off the front of the data structure very frequently, copying to a deque before this processing may be an option. This costs you a copy and deque memory isn't contiguous, so you can't just pass a pointer to a C API.

Bottom line, learn about the data structures and their tradeoffs before diving in, however vector of char is typically what I see used in general practice.

Solution 2:

The biggest problem with std::string is that the current standard doesn't guarantee that its underlying storage is contiguous. However, there are no known STL implementations where string is not contiguous, so in practice it probably won't fail. In fact, the new C++0x standard is going to fix this problem, by mandating that std::string uses a contiguous buffer, such as std::vector.

Another argument against string is that its name suggests that it contains a character string, not a binary buffer, which may cause confusion to those who read the code.

That said, I recommend vector as well.