Detect spammers on my server

I recently got one Undelivered Mail Returned to Sender while sending my newsletter to one of my 1500 customers. My website uses a double-opt-in procedure to make sure, the user explicitly want to receive my newsletter.

The error message:

smtp; 554 ...
    Swisscom AG IP: 94.130.34.42, You are not allowed to send us mail. Please
    refer to xyz.com if you feel this is in error.

I got an example spam mail (from the mail provider of the receiving mailserver):

Received: from mail.com ([94.130.34.42])
        by smtp-27.iol.local with SMTP
        id itOWeYZ6O42IFitOWe35TR; Tue, 13 Feb 2018 03:54:09 +0100
From: "Servizi online - Poste Italiane" <[email protected]>
Subject: Abbiamo ricevuto una segnalazione di accredito
Date: Mon, 12 Feb 2018 11:32:03 -0500

The provider also stated, that my server seems to be hacked. He further stated, that "the recipient mail server has simply recorded the rDNS presented to it by the connecting IP, in this case mail.com ([94.130.34.42])" - which is definitely NOT as I configured my rDNS entry (mail.lotsearch.de) for my IP address. So if I understood rDNS correctly, the receving mail server queries the sender IP for a rDNS entry (94.130.34.42 => should resolve to => mail.lotsearch.de, which it definitely does, when I test it from my local machine via $ host 94.130.34.42).

How is it possible to spoof rDNS? I can not imagine any way how this can technically work (only with a man-in-the-middle attack somewhere in the infrastructure between the receiving mailserver and my server).

The provider mentioned also, that "it is likely that a machine connecting from my IP has been compromised and sending these messages via direct connections to the recipient mail sever (also known as direct MX)". What does direct MX mean? Someone stole or found leaked mail credentials to one of my mail accounts and used them for mail sending?

What I have done so far to make sure my server is NOT / will not be hacked:

• searched the mail logs (var/log/mail*): nothing special in there
• checked the ssh login logs (last, lastb): nothing unusual
• checked if postfix is does relaying: no it does not (checked via telnet)
• checked for malware via clamav: no results
• installed and configured fail2ban for ssh, postfix and dovecot
• installed the latest patches / updates for Ubuntu 16.04 (I do so every week)
• checked if my IP address is on any blacklist: it is not
• verified rDNS entry in the management console of my hosting provider: it is correctly set to mail.lotsearch.de.
• changed passwords of all mail accounts
• changed public keys for shell access

More important: There was no information about [email protected] in the logs. So if my server would have been misused by a spammer (f.e. because of leaked smtp credentials of one of the mail accounts) I would see that in the log files.

The last possibility I can think of is that an intruder placed malware at my server I didn't find yet.

How can I monitor outgoing mail traffic (per process and per port)?

Only monitoring outgoing port 25 would not help as this would only trap irregular mails sent via postfix, but not the mail traffic caused by a potential malware infection (if the malware uses another port than 25 for directly sending mails / communicating with recipient mail servers). If I monitor outgoing traffic on all ports I will get a way to huge log file which I cannot search through for suspicious activity efficiently.

EDIT - Added test for open relay:

$ telnet mail.lotsearch.de 25
$ HELO [email protected]
250 mail.lotsearch.de
$ MAIL FROM: [email protected]
250 2.1.0 Ok
$ RCPT TO:<[email protected]>
454 4.7.1 <[email protected]>: Relay access denied

EDIT - Running webapps

• Custom Platform based on Zend Framework 3 (https://framework.zend.com/)
• Mediawiki (https://www.mediawiki.org/)
• Mantis Bug Tracker (https://www.mantisbt.org/)

Before I get to my suggestion, I want to comment a bit on something your provider said to you:

 Received: from mail.com ([94.130.34.42])

        by smtp-27.iol.local with SMTP
        id itOWeYZ6O42IFitOWe35TR; Tue, 13 Feb 2018 03:54:09 +0100

This does not indicate that the reverse DNS for 94.130.34.42 is (or was) mail.com. Rather, it indicates that the SMTP client sent mail.com in its HELO (or EHLO) line. (A well-configured mail server would have rejected this connection entirely, but that's on Swisscom, not you...) This line does not indicate any reverse DNS entry. If it did, it would have appeared within the parentheses. For example:

Received: from mail-io0-f197.google.com (mail-io0-f197.google.com [209.85.223.197])

In this case, the first hostname is what the mail server identified itself as in its EHLO. The second hostname is the reverse DNS recorded at the time the connection was made.

RFC 5321 section 4.4 explains the format of the Received: line, with a formal grammar.

In your case, no reverse DNS was recorded. Since your IP address has a PTR record, this may be because they didn't look it up, or there was a temporary DNS failure.

Now, it appears you run a web hosting service and have numerous web apps. If one of these is compromised, it may begin sending spam. These often make direct connections to remote mail servers by looking up their MX records and connecting to port 25, as if they were actually a mail server themselves, rather than delivering mail to the local mail spool or an authenticated mail service on ports 587 or 465 as legitimate web apps do.

One way I stop this is by implementing a firewall rule that prevents outgoing connections on port 25 unless the user is the mail server user. For example:

iptables -I OUTPUT -m owner ! --uid-owner postfix -m tcp -p tcp --dport 25 -j REJECT

Web apps can no longer deliver mail directly to remote SMTP servers, but must use the local mail spool or an authenticated mail service.

In this day and age, trying to do your own mail server is, for the most part, a loosing battle and you are better off finding an affordable service. Having said that..

• Look at your logs going to the provider that blocked you and see if you can find anything suspicious. It is possible, and happens often, that someone forgets they subscribed to your newsletter and marks you as spam. Then depending on the provider you can get on the provider's blacklist even though you have done nothing wrong.

• Separate mass mailings from all your other email into two servers.

• Keep logs for weeks at a minimum and better months. So anytime something happens you research.

• Check your logs daily for similar situations from any provider and look into it daily, or faster.. The second you get blocked and if you keep trying to send it can get worse. You can go from a temporary block to a permanent block.. to getting reported to a blacklist.

• Not sure how they implement it, but one thing I know many providers do for outbound mail services is that the second a provider/IP blocks an email then no other emails get tried to be sent. Ideally you want something like that. Because the second one gets blocked, sending more will just aggravate the issue.