OpenGL ES2 Alpha test problems

Here is an overview of a few methods to render with transparency in OpenGL, with advantages and disadvantages for each.

Alpha Testing

This is a very limited method, but is sufficient for the specific case the poster asked about. The example shown does not really need transparency because everything is either fully opaque or fully transparent (alpha = 1.0 or alpha = 0.0).

There used to be an alpha test for this purpose in OpenGL, but that is a deprecated feature, and is of course not in ES. You can emulate the same thing in your fragment shader, which will look something like this:

vec4 val = texture2D(tex, texCoord);
if (val.a > 0.5) {
    gl_FragColor = val;
} else {
    discard;
}

Advantages:

• Simple.
• No additional work on app side.

Disadvantages:

• Only works for full opacity/transparency, cannot deal with semi-transparency.
• Can hurt performance because it typically means that depth testing before the fragment shader has to be disabled.

Sorting and Blending

Rendering transparency is a primary use case for blending. The most common approach is to set the blend function toSRC_ALPHA, ONE_MINUS_SRC_ALPHA, enable blending, and render with the alpha component of the rendered fragments containing the desired opacity.

If the scene contains a mixture of fully opaque objects and objects with transparency, the fully opaque objects can be rendered first, without a need for them to be sorted. Only the objects with transparency need to be sorted. The sequence is then:

1. Render fully opaque geometry.
2. Render non-opaque geometry, sorted back to front.

Advantages:

• Can handle semi-transparency.
• Can handle multiple layers of transparent geometry.
• Rendering itself is very efficient.

Disadvantages:

• Requires sorting for correct result. For the blend function mentioned above, geometry has to be rendered back to front. Depending on the application, this can be between no big deal and almost impossible. For example, to correctly render intersecting geometry, you may have to start splitting triangles, which is far from attractive.

Depth Peeling

This is a very clever use of OpenGL features, IMHO, and can be a good practical solution. It does require multiple rendering passes. The simple form requires 3 passes:

1. Render scene with the usual settings (depth testing enabled, depth function LESS, color and depth write enabled), but render only the fully opaque geometry. If opacity is per object, you can handle that by skipping draw calls for non-opaque objects. Otherwise, you will have to discard non-opaque fragments with a shader similar to the one under Alpha Testing above.
2. Render the non-opaque geometry with the same settings as above, except that color write is disabled.
3. Render the non-opaque geometry again, but this time with depth function EQUAL, color write enabled again, depth write disabled, and using blending.

A minimal shader can be used for pass 2, since it does not need to produce any valid fragment colors.

Advantages:

• Simple to implement.
• Reasonably efficient, does not require sorting.
• Correctly handles semi-transparency.

Disadvantages:

• Simple form only draws front-most layer of transparent geometry. This may sound like a major limitation, but the results can actually look very good. There are more advanced forms where additional layers are rendered with additional passes. Beyond the overhead for the those additional passes, it also gets more complex because it requires multiple depth buffers. I believe there's a white paper about it on the NVIDIA web site.

Alpha to Coverage

I haven't used this myself, so the following is based on my limited theoretical understanding. It looks like another interesting method. This requires multi-sampled rendering. The feature is enabled with glEnable(GL_SAMPLE_ALPHA_TO_COVERAGE), which then translates alpha values into a coverage mask, resulting in only parts of the samples being written depending on alpha value. This results in a transparency effect when the multi-sample buffer is downsampled to the final color buffer.

Advantages:

• Can handle semi-transparency.
• Correctly handles multiple layers of transparency.
• Efficient, particularly if MSAA would have been used anyway. No sorting required.

Disadvantages:

• Requires MSAA. Modern GPUs are very effective at MSAA rendering, so this is no huge deal. Often times, you will probably want to use MSAA anyway.
• Effective resolution of alpha value is very small, unless I'm missing something. For example, with 4x MSAA, you can only represent 5 possible alpha values (0, 1, 2, 3, 4 samples set in coverage mask).