Best way to combine the output of 2 neural networks?

Solution 1:

The best results I have achieved is to actually use 3 models. I take as the final prediction the case where A: all of the models predict the same class or if that is not the case B: select the class where 2 of the 3 models agree. Since you are doing binary classification one of these will always be the case.