How do I "subtract" a color filter using GPUImageLibrary?

Color matrix is the entity that defines new color component values for some pixel as linear functions of the current color components of the same pixel. I.e. the only input for color matrix conversion is the pixel, which colors should be transformed. There is no way to involve another pixels to such conversion. Regardless if they are pixels from another image or even neighbors of transformed pixel - it is impossible.

Disclaimer

Originally I answered another question from the same author and it seems was asked within the same problem. I decided not to mark this one as a duplicate, because the first question doesn't have context of using the GPUImageLibrary, which makes the things way simpler. However I don't want to overact, so if you need any additional details, please refer to the original answer. Here I'll merely outline the solution.

You don't actually need to extend the GPUImageSubtractBlendFilter or any descendants of the GPUImageTwoInputFilter, because (as correctly was noted by Sergio) you should deal with only one instance of texture. Instead define your own fragment shader that will take a color filter, apply it and subtract the resulting value from original pixel:

precision mediump float;
struct ColorFilter {
  mat4 factor;
  vec4 shift;
};
uniform sampler2D inputImageTexture;
uniform ColorFilter uColorFilter;
varying vec2 textureCoordinate;
void main() {
  vec4 originalColor = texture2D(inputImageTexture, textureCoordinate);
  originalColor.rgb *= originalColor.a;
  vec4 filteredColor = (originalColor * uColorFilter.factor) + uColorFilter.shift;
  filteredColor.rgb *= filteredColor.a;
  gl_FragColor = vec4(originalColor.rgb - filteredColor.rgb, originalColor.a);
}

And now just attach your color filter matrix to the shader:

@Override
public void onInitialized() {
    super.onInitialized();
    attachColorFilter(getProgram());
}

// ========================================== //
// Private
// ========================================== //

private void attachColorFilter(int program) {
    final float[] colorFilterFactor = new float[4 * 4];
    final float[] colorFilterShift = new float[4];
    for (int i = 0; i < mColorFilter.length; i++) {
        final float value = mColorFilter[i];
        final int calculateIndex = i + 1;
        if (calculateIndex % 5 == 0) {
            colorFilterShift[calculateIndex / 5 - 1] = value / 255;
        } else {
            colorFilterFactor[i - calculateIndex / 5] = value;
        }
    }
    final int colorFactorLocation = GLES20.glGetUniformLocation(program,
            "uColorFilter.factor");
    setUniformMatrix4f(colorFactorLocation, colorFilterFactor);
    final int colorShiftLocation = GLES20.glGetUniformLocation(program,
            "uColorFilter.shift");
    setFloatVec4(colorShiftLocation, colorFilterShift);
}

Feel free to use the gist with the working subclass. Just instantiate it and pass to the GPUImageView:

mBlendedImageView.setFilter(new ColorMatrixSubtractFilter(new float[] {
    0.393f, 0.7689999f, 0.18899999f, 0, 0,
    0.349f, 0.6859999f, 0.16799999f, 0, 0,
    0.272f, 0.5339999f, 0.13099999f, 0, 0,
    0,      0,         0,            1, 0
}));

On the right you can see the output of subtracted filter applied to the image on the left: