Forwarding email with postfix via AWS SES

email postfix amazon-ses smtp-headers mail-forwarding

I have a server with postfix MTA installed that accepts mail for my domain, hereinafter @example.com. I need to setup forwarding for certain [email protected] to other emails . e.g. @gmail.com and this needs to be done using SMTP transport via SES.

I have SES production access and login/password pair, that is active and has IAM permissions, I checked. I have confirmed certain From: address at SES. If I send email with these SMTP username/password directly via SES from a PHP script, everything works fine. This means that the login/password pair and From: are alright.

Now I need to rewrite From: for the emails that are going from outside and need to be forwarded. For example, an email is sent from @gmail.com -> [email protected]. The entry user is set up in virtual_alias_maps and should go to another @gmail.com. In order to rewrite From: header, the following smtp_header_checks is used:

smtp_header_checks = pcre:/etc/postfix/header_checks

the content of /etc/postfix/header_checks is following: 

/^From:(.*)/ REPLACE From: "$1" <[email protected]>

[email protected] is confirmed at SES. This is the same email I'm using when testing with PHP script that uses SES directly, i.e. with no header rewrite.

When I send something from @gmail to [email protected], here is what's going on in the logs:

Apr  8 15:04:05 ip-10-191-106-25 postfix/smtpd[32545]: connect from mail-wg0-f42.google.com[74.125.82.42]
Apr  8 15:04:06 ip-10-191-106-25 postfix/smtpd[32545]: 3252A2415D: client=mail-wg0-f42.google.com[74.125.82.42]
Apr  8 15:04:06 ip-10-191-106-25 postfix/cleanup[32550]: 3252A2415D: message-id=<CADLOpCq4ZFR5=xqTNxPO73-tVkF63urLt_9ueGBDrLSggy29MQ@mail.gmail.com>
Apr  8 15:04:06 ip-10-191-106-25 postfix/qmgr[32192]: 3252A2415D: from=<[email protected]>, size=1687, nrcpt=1 (queue active)
Apr  8 15:04:06 ip-10-191-106-25 postfix/smtpd[32545]: disconnect from mail-wg0-f42.google.com[74.125.82.42]
Apr  8 15:04:07 ip-10-191-106-25 postfix/smtp[32551]: 3252A2415D: replace: header From: "User" <[email protected]>: From: " "User" <[email protected]>" <[email protected]>
Apr  8 15:04:07 ip-10-191-106-25 postfix/smtp[32551]: 3252A2415D: to=<[email protected]>, orig_to=<[email protected]>, relay=email-smtp.us-west-2.amazonaws.com[54.149.142.243]:25, delay=1.3, delays=0.22/0.03/0.57/0.43, dsn=5.0.0, status=bounced (host email-smtp.us-west-2.amazonaws.com[54.149.142.243] said: 554 Message rejected: Email address is not verified. (in reply to end of DATA command))
Apr  8 15:04:07 ip-10-191-106-25 postfix/cleanup[32550]: 7FD75243F8: message-id=<[email protected]>
Apr  8 15:04:07 ip-10-191-106-25 postfix/qmgr[32192]: 7FD75243F8: from=<>, size=3700, nrcpt=1 (queue active)
Apr  8 15:04:07 ip-10-191-106-25 postfix/bounce[32552]: 3252A2415D: sender non-delivery notification: 7FD75243F8
Apr  8 15:04:07 ip-10-191-106-25 postfix/qmgr[32192]: 3252A2415D: removed
Apr  8 15:04:08 ip-10-191-106-25 postfix/smtp[32551]: 7FD75243F8: to=<[email protected]>, relay=email-smtp.us-west-2.amazonaws.com[54.69.81.169]:25, delay=0.67, delays=0.01/0/0.59/0.07, dsn=5.0.0, status=bounced (host email-smtp.us-west-2.amazonaws.com[54.69.81.169] said: 501 Invalid MAIL FROM address provided (in reply to MAIL FROM command))
Apr  8 15:04:08 ip-10-191-106-25 postfix/qmgr[32192]: 7FD75243F8: removed

That gives me an idea that the From: header was not actually rewritten. However, the line:

Apr  8 15:04:07 ip-10-191-106-25 postfix/smtp[32551]: 3252A2415D: replace: header From: "User" <[email protected]>: From: " "User" <[email protected]>" <[email protected]>

tells us it actually has been.

I realize that I need to view the rewritten whole mail body before it's sent to AWS server, but I have no idea how to debug it.

Here is the main.cf contents:

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = example.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = example.com, ip-10-191-106-25.ec2.internal, localhost.ec2.internal, localhost, domain2.example.com
relayhost = email-smtp.us-west-2.amazonaws.com
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
default_transport = smtp
relay_transport = relay

smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl_password
smtp_use_tls = yes
smtp_tls_security_level = encrypt
smtp_tls_note_starttls_offer = yes

smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

smtp_header_checks = pcre:/etc/postfix/header_checks

virtual_alias_maps = hash:/etc/postfix/virtual

sender_canonical_maps = hash:/etc/postfix/canonical

#debug_peer_list=email-smtp.us-west-2.amazonaws.com 127.0.0.1
#debug_peer_level=5

Solution 1:

Looks like your initial goal with header_checks is preserving original sender and replace it with allowed sender of Amazon SES.

The problem of your approach above is the From: header become non-standard because of multiple quotes in there.

From: " "User" <[email protected]>" <[email protected]>

Daniel R. Tobias mentioned this issue in his article: Dan's Mail Format Site | Headers | From/To/CC/BCC

One thing that will put you at risk of having your mail program inflict nonstandard header lines on your messages is to attempt to include quotation marks within your name, like Jesse "The Body" Ventura. If inserted directly into the header, within double quotes, you'd get "Jesse "The Body" Ventura", which actually parses into two quoted strings, "Jesse " and " Ventura", with The Body sitting in the middle with uncertain purpose.

So you can rely on this nonstandard header to bypass Amazon SES checker.

One approach to solve this problem is split the two goals above in two header_checks, header_checks and smtp_header_checks. The first header_checks will preserve the original sender in another custom header (for example X-Original-From). The second one will replace the From: header.

#main.cf
header_checks = pcre:/etc/postfix/first_header_checks
smtp_header_checks = pcre:/etc/postfix/second_header_checks

#first_header_checks
/^From:(.*)/ PREPEND X-Original-From: $1

#second_header_checks
/^From:(.*)/ REPLACE From: <[email protected]>

Because of this schema, X-Original-From: header will be added in every incoming email. But replacing action will be executed on outgoing email only.

Another way is using pcre to exclude quote in the original From: header. Unfortunately, I don't have any time to test some ideas right now. Maybe later... I'll update this answer with other workaround.

The envelope sender address rewriting

The above parts is still half-journey. To pass amazon SES, you need rewrite the envelope sender address too.

With the exception of addresses containing labels (see below), you must verify each email address (or the domain of the email address) that you will use as a "From" or "Return-Path" address for your messages. Until your account is out of the Amazon SES sandbox, you must also verify the email address of every recipient except for the recipients provided by the Amazon SES mailbox simulator.

See the differences between Envelope Address vs. Message Header Address in this article.

I've explained some steps to rewrite the sender in this similar threads: AWS SES: "Email address is not verified" error with Postfix relay. Basically you need put this parameter in main.cf

sender_canonical_maps = regexp:/etc/postfix/sender_canonical
sender_canonical_classes = envelope_sender
smtpd_data_restrictions = check_sender_access pcre:/etc/postfix/sender_access

In /etc/postfix/sender_canonical, add

/.*/    [email protected]

In /etc/postfix/sender_access, add

/(.*)/  prepend X-Envelope-MailFrom: <$1>

The /etc/postfix/sender_access is used to preserved the original envelope sender address.

Solution 2:

following this step its working for me able to fordward the email from inside the server to customer -> 

sender_canonical_maps = regexp:/etc/postfix/sender_canonical
sender_canonical_classes = envelope_sender
smtpd_data_restrictions = check_sender_access pcre:/etc/postfix/sender_access

In /etc/postfix/sender_canonical add

/.*/    [email protected]

In /etc/postfix/sender_access add

/(.*)/  prepend X-Envelope-MailFrom: <$1>

Related

Only allow ssh connections to a specific domain
Clearer understanding for glusterfs for replication
Has windows XP support multiple remote login session (RDP) at a time?
Buy vs rent for servers and other hardware
Using a package manager vs compiling from source for latest updates
Is there a way to set up an Exchange mailbox to collect email "anonymously"?
Qmail Patching Makes me Nervous
What is the maximum memory a process (MySQL) can consume on a 32-bit OS?
Reset postgresql
catch forEach last iteration
creating list of objects in Javascript
Is there a timeout for idle PostgreSQL connections?