How can Postfix filter email (DKIM) without keeping the message in memory and without writing it to disc twice?

Solution 1:

Postfix queue files are not flat mails. Adding a header does not require a rewrite. To take advantage of that use the milter interface. The answers I got from postfix-users make me believe mail is not kept in memory during milter processing either. At least not by Postfix.

Using the pipe mechanism with the after-queue content filter would not do it as mentioned in the question. A write out to file to avoid the mail in memory would probably be reasonable enough though and better than keeping it in memory.

While the milter interface is good enough for DKIM, I’ld like to list it’s shortcomings (all of them could have been avoided):

• you can not modify neither header nor body(-parts) before the entire message is received
  • no proper in transit piping
• you can not back reference no header and no body once you are allowed to replace/modify content
  • milter client needs to keep a copy during reception phase if it needs the information
• the body can only be replaced in its entirety
• header substitution/deletion require name and index, but milter server is not passing the index number (or any other opaque unique reference)
  • milter client needs to count headers for any header it might later decide to replace

Postfix has some shortcomings as well:

• postfix offers 3 filter mechanisms at 2 positions
  • you can not mix and match mechanism and position
  • the most appropriate mechanism for DKIM is milter
  • the most appropriate place for DKIM signing is after queue
  • after queue milter is not available
    • within limits that would be possible
    • postfix can actually already fake SMTP/milter environments to make milters work in new areas (“non-smtp-milter”)
• no mechanism exploits all benefits of what would be possible with the current queue data structure
  • not needed for DKIM, though; just saying