SSE, intrinsics, and alignment

Solution 1:

First of all you have to care for two types of memory allocation:

• Static allocation. For automatic variables to be properly aligned, your type needs a proper alignment specification (e.g. __declspec(align(16)), __attribute__((aligned(16))), or your _MM_ALIGN16). But fortunately you only need this if the alignment requirements given by the type's members (if any) are not sufficient. So you don't need this for you Sphere, given that your Vector3 is already aligned properly. And if your Vector3 contains an __m128 member (which is pretty likely, otherwise I would suggest to do so), then you don't even need it for Vector3. So you usually don't have to mess with compiler specific alignment attributes.

• Dynamic allocation. So much for the easy part. The problem is, that C++ uses, on the lowest level, a rather type-agnostic memory allocation function for allocating any dynamic memory. This only guarantees proper alignment for all standard types, which might happen to be 16 bytes but isn't guaranteed to.

  For this to compensate you have to overload the builtin operator new/delete to implement your own memory allocation and use an aligned allocation function under the hood instead of good old malloc. Overloading operator new/delete is a topic on its own, but isn't that difficult as it might seem at first (though your example is not enough) and you can read about it in this excellent FAQ question.

  Unfortunately you have to do this for each type that has any member needing non-standard alignment, in your case both Sphere and Vector3. But what you can do to make it a bit easier is just make an empty base class with proper overloads for those operators and then just derive all neccessary classes from this base class.

  What most people sometimes tend to forget is that the standard allocator std::alocator uses the global operator new for all memory allocation, so your types won't work with standard containers (and a std::vector<Vector3> isn't that rare a use case). What you need to do is make your own standard conformant allocator and use this. But for convenience and safety it is actually better to just specialize std::allocator for your type (maybe just deriving it form your custom allocator) so that it is always used and you don't need to care for using the proper allocator each time you use a std::vector. Unfortunately in this case you have to again specialize it for each aligned type, but a small evil macro helps with that.

  Additionally you have to look out for other things using the global operator new/delete instead of your custom one, like std::get_temporary_buffer and std::return_temporary_buffer, and care for those if neccessary.

Unfortunately there isn't yet a much better approach to those problems, I think, unless you are on a platform that natively aligns to 16 and know about this. Or you might just overload the global operator new/delete to always align each memory block to 16 bytes and be free of caring for the alignment of each and every single class containing an SSE member, but I don't know about the implications of this approach. In the worst case it should just result in wasting memory, but then again you usually don't allocate small objects dynamically in C++ (though std::list and std::map might think differently about this).

So to sum up:

• Care for proper alignment of static memory using things like __declspec(align(16)), but only if it is not already cared for by any member, which is usually the case.

• Overload operator new/delete for each and every type having a member with non-standard alignment requirements.

• Make a cunstom standard-conformant allocator to use in standard containers of aligned types, or better yet, specialize std::allocator for each and every aligned type.

Finally some general advice. Often you only profit form SSE in computation-heavy blocks when performing many vector operations. To simplify all this alignment problems, especially the problems of caring for the alignment of each and every type containing a Vector3, it might be a good aproach to make a special SSE vector type and only use this inside of lengthy computations, using a normal non-SSE vector for storage and member variables.

Solution 2:

Basically, you need to make sure that your vectors are properly aligned because SIMD vector types normally have larger alignment requirements than any of the built-in types.

That requires doing the following things:

1. Make sure that Vector3 is properly aligned when it is on the stack or a member of a structure. This is done by applying __attribute__((aligned(32))) to Vector3 class (or whatever attribute is supported by your compiler). Note, that you don't need to apply the attribute to structures containing Vector3, that is not necessary and not enough (i.e. no need to apply it to Sphere).

2. Make sure that Vector3 or its enclosing structure is properly aligned when using heap allocation. This is done by using posix_memalign() (or a similar function for your platform) instead of using plain malloc() or operator new() because the latter two align memory for built-in types (normally 8 or 16 bytes) which is not guaranteed to be enough for SIMD types.

Solution 3:

1. The problem with the operators is that by themselves they're not sufficient. They don't affect stack allocations, for which you still need __declspec(align(16)).

2. __declspec(align(16)) affects how the compiler places objects in memory, if and only if it has the choice. For new'ed objects, the compiler has no choice but to use the memory returned by operator new.

3. Ideally, use a compiler which handles them natively. There is no theoretical reason why they need to be treated differently from double. Else, read the compiler documentation for workarounds. Each handicapped compiler will have its own set of issues and therefore its own set of workarounds.