Postfix bounce sent twice (double-bounce)
Now that I understand what was happening the answer is obvious.
Following the RFC the MTA (should) always send a bounce message to the sender (FROM or Return Path if set).
Adding the notify_classes does not change this behaviour, it adds behaviour on top of it. (So a 2nd mail is sent)
To achieve what I wanted, that only the script gets called, I removed the notify_classes and bounce_notice_recipient. Plus, I changed our application to always add a "Return-Path" header with [email protected]
When a bounce happens, the MTA sends a bounce message to the FROM or the Return Path if it has been set (which is now [email protected]). As this email is mapped in the transport_maps it get's piped to the script and no emails are actualy sent. Hooray