Never give in except to convictions of honour and good sense
The difference between give in and give up is subtle. To give up is to forfeit, lose, or acquiesce, like letting a ball fall from your hands. To give in is to yield or submit to pressure, like a paper cup crumpling when you squeeze it. Giving up is more like something that you do, while giving in is more like allowing something to happen to you. In most situations, they’re freely interchangeable.
The whole phrase is saying that you should only yield to pressure when either your sense of honour, or your sense of reason, says that you should. You should not give in—and thus compromise your honour or reason—under pressure from your enemies.