Barriers in OpenCL

In OpenCL, my understanding is that you can use the barrier() function to synchronize threads in a work group. I do (generally) understand what they are for and when to use them. I'm also aware that all threads in a work group must hit the barrier, otherwise there are problems. However, every time I've tried to use barriers so far, it seems to result in either my video driver crashing, or an error message about accessing invalid memory of some sort. I've seen this on 2 different video cards so far (1 ATI, 1 NVIDIA).

So, my questions are:

1. Any idea why this would happen?
2. What is the difference between barrier(CLK_LOCAL_MEM_FENCE) and barrier(CLK_GLOBAL_MEM_FENCE)? I read the documentation, but it wasn't clear to me.
3. Is there general rule about when to use barrier(CLK_LOCAL_MEM_FENCE) vs. barrier(CLK_GLOBAL_MEM_FENCE)?
4. Is there ever a time that calling barrier() with the wrong parameter type could cause an error?

Solution 1:

As you have stated, barriers may only synchronize threads in the same workgroup. There is no way to synchronize different workgroups in a kernel.

Now to answer your question, the specification was not clear to me either, but it seems to me that section 6.11.9 contains the answer:

CLK_LOCAL_MEM_FENCE – The barrier function will either flush any variables stored in local memory or queue a memory fence to ensure correct ordering of memory operations to local memory.

CLK_GLOBAL_MEM_FENCE – The barrier function will queue a memory fence to ensure correct ordering of memory operations to global memory. This can be useful when work-items, for example, write to buffer or image memory objects and then want to read the updated data.

So, to my understanding, you should use CLK_LOCAL_MEM_FENCE when writing and reading to the __local memory space, and CLK_GLOBAL_MEM_FENCE when writing and readin to the __global memory space.

I have not tested whether this is any slower, but most of the time, when I need a barrier and I have a doubt about which memory space is impacted, I simply use a combination of the two, ie:

barrier(CLK_LOCAL_MEM_FENCE | CLK_GLOBAL_MEM_FENCE);

This way you should not have any memory reading\writing ordering problem (as long as you are sure that every thread in the group goes through the barrier, but you are aware of that).

Hope it helps.

Solution 2:

Reviving an old-ish thread here. I have had a little bit of trouble with barrier() myself.

Regarding your crash problem, one potential cause could be if your barrier is inside a condition. I read that when you use barrier, ALL work items in the group must be able to reach that instruction, or it will hang your kernel - usually resulting in a crash.

if(someCondition){
  //do stuff
  barrier(CLK_LOCAL_MEM_FENCE);
  //more stuff
}else{
  //other stuff
}

My understanding is that if one or more work items satisfies someCondition, ALL work items must satisfy that condition, or there will be some that will skip the barrier. Barriers wait until ALL work items reach that point. To fix the above code, I need to restructure it a bit:

if(someCondition){
  //do stuff
}
barrier(CLK_LOCAL_MEM_FENCE);
if(someCondition){
  //more stuff
}else{
  //other stuff
}

Now all work items will reach the barrier.

I don't know to what extent this applies to loops; if a work item breaks from a for loop, does it hit barriers? I am unsure.

UPDATE: I have successfully crashed a few ocl programs with a barrier in a for-loop. Make sure all work items exit the for loop at the same time - or better yet, put the barrier outside the loop.

(source: Heterogeneous Computing with OpenCL Chapter 5, p90-91)