Our IP was listed by Spamhaus - how can we prevent this from happening again?

I'm the technical assistant at a small-town public library where we run our own e-mail. If it matters, it's Postfix on an Ubuntu 18.04 box. Gateway / firewall is a 16.04 box with port forwarding set up through a script that runs at boot, issuing a long sequence of iptables commands.

Today our IP address got onto two Spamhaus block lists, first the SBL and then also the XBL. Last week, after we had an outgoing message bounce, I saw we were on the SBL list. I went through the process to remove it and it was removed. But the problem came roaring back this afternoon, and the web server at Spamhaus pretended to accept my requests to remove the IP, but it stayed on the list and after a few runs it also turned up on the XBL.

I called our internet provider and their network engineer went through the removal steps and actually got it removed. (Or perhaps my first request took that long to process.)

Where should I go from here? I am only a novice systems administrator and what I don't know could fill a... library. To prevent this from happening again, I need to find out what, if anything, is sending spam. I've tried tcpdump on the external NIC for the gateway watching port 25 but I didn't see anything suspicious (to me) during the time I looked (after the fact, and it wasn't long before I moved on to something else, I admit). But I don't even know what's going in and what's going out, and I'm in way over my head. I installed Wireshark on the gateway but soon removed it because it seemed to require a graphical desktop to use. I looked at /var/log/mail.log but I don't see anything that looks like bulk mail going out (but what would that look like?).

Am running sudo tcpdump -i enp3s0 port 25 | tee feb26-27-overnight.log overnight (on the gateway, monitoring the external-facing NIC) in hopes of finding something to glom onto. But it would be nice if there were a straightforward way to find out what's going on without requiring a cyberforensic investigation just to be able to send e-mail to people.

Sequel! How do I ensure that only my e-mail server is sending e-mail through my network?

Where should I go from here?

I'm not trying to be glib or dismissive, but you ought to move your email to a professionally hosted service. Office 365. GSuite. Whatever. There's no good reason your organization should be hosting your email in house in 2020. You can't provide the level of availability, scalability, and reliability that these services can... not too mention your current email reputation problem, which you won't have with these services.

The only valid reasons I can think of for your continuing to do this yourself are either budgetary issues or legal issues which would prevent you from moving your email "out of house".

What I've done so far is get everything set up for SPF, DKIM, and finally DMARC. Implementation details are going to vary by network and by platform, so I won't give a blow-by-blow of absolutely everything I did. But having done it all seems to have greatly improved the acceptance of my mail.

SPF: Pretty low-hanging fruit. All you have to do is set up a DNS record saying who's allowed to send mail for you, and there are lots of resources to help you compose it, e.g. SPF Wizard.

DKIM: Relatively complex. You have to deal with keys and a bunch of other server-side stuff, then set up a corresponding DNS record. I followed these instructions for Ubuntu (using Postfix and Dovecot) on LinuxBabe. This allows e-mails to be checked for authenticity, that the sender wasn't just spoofing the origin to pretend to be from you.

DMARC: Simple, but you need to have at least one of SPF and DKIM set up first. Similar to SPF, you're just making a DNS record that tells third parties what to do with your mail if it doesn't pass SPF and/or DKIM. General wisdom is to start with "p:none" and read reports for a while until you're satisfied that all your outgoing mail (which might be from more than one source - e.g., we also use Sendgrid), is passing. (You can get daily digests from Gmail, for example.) Then you can ratchet your recommendation up to "p:quarantine" or "p:reject". Some basic info here.

Oh, and a quick do-not-do: Don't use virtual domain aliases or whatever they're called for forwarding purposes. They're not only awkward to use (I had to to spell them out for every single alias a user might have, although there might be another way), but also I think they forward messages before they go through any of your own spam-filtering rules. We got blocked by Outlook almost immediately after forwarding one of our staff's email to their Hotmail account. Instead, since in our setup server users = mail users, we can just set up a .forward file in the user's home directory, with contents like so: [email protected], \localusername. This is far more elegant and kicks in only after a message actually reaches a mailbox.