Why do I need an SMTP server?

Why do I need an intermediate SMTP server to send mail? Why can’t my client (Outlook, Thunderbird) send messages directly to the recipient’s SMTP domain?

For example, if I have to send an email to [email protected] with my Gmail account, I send it to the smtp.gmail.com server; and then this server will send my message to the MX server of example.com.

Solution 1:

It is technically possible to send an email directly to the recipient's SMTP server from your computer.

Looking at it from a historical basis, if the remote SMTP server is down, you want a system to automatically handle it and keep retrying it— hence you have an SMTP server. Similarly, in the old days, not all mail servers were connected all the time—long distance links were expensive, so mail would be queued and sent when a link was established.

Moving on to where Internet is cheap, it's still useful to have mechanisms to retry sending email if a server is unavailable, and it's not ideal for this functionality to be written into the MUA (Mail user agent/end user mail program). These functions fit into an MTA (Mail server/SMTP server).

But it gets worse—spammers. Most emails (way more than 80%) are spam. So mail providers do whatever they can to reduce this problem—and a large number of techniques make assumptions about the way email is delivered—the following are important considerations:

1. Greylisting: Some providers will automatically drop a mail connection if the sender and recipient have not communicated before, and expect them to try a second time—because spammers often don't, while an SMTP server is always supposed to. This reduces spam volumes by about 80%. It sucks to have to do this though.

2. Reputation: It is a lot more likely that someone sending email through a reputable, known SMTP server is legit than a fly-by-night server. To get a feel for reputation, providers do a number of things:

   1. Block dynamic / client addresses (Not 100%, but large chunks of the Internet have been mapped out).

   2. Look that reverse DNS matches forward DNS: Not very hard to do, but shows some level of accountability and knowledge of best practices—and something a lot of client address blocks don't have.

   3. Reputation: When communicating with other SMTP servers, a lot of providers keep track of the amount of spam and volumes of email sent and can reduce the amount of spam by limiting connections and keeping an eye on these parameters. (There are lots of ways this is done, not all of them obvious, but which require a known sender).

   4. SPF and DKIM: These mechanisms tie DNS resources to the domain name to make forging mail harder, and would be difficult (but not necessarily impossible to deploy if the mail program (MUA) is responsible for outgoing mail. (Added to make this answer more complete as it's already been accepted. Credit for it should go to posters below as it slipped my mind, but is, nonetheless, very valid)

There are probably other minor concerns, but these would be the major ones.

Solution 2:

Why do I need an intermediate SMTP server to send mail? Why can’t my client (Outlook, Thunderbird) send messages directly to the recipient’s SMTP domain?

In 1991 — and most of the early 1990s and even earlier — you might be able to do what you describe. But the reality in 2015 is, while one can technically send an email to anyone from any machine that has a mail service installed on it, the world of SPAM has made that method effectively useless.

When you use a “real” SMTP service, things are set like PTR records, SPF records and even DomainKeys that are all established for one purpose and one purpose only: to vouch that the SMTP that is sending the message is legit. And if it isn’t? Filter the message into a SPAM folder or the “great abyss” of deletion. Here is a breakdown of what each of those items are:

• PTR (Pointer Record/Reverse DNS Record): Server level verification. As explained here, a PTR record is used to map a network interface (IP) to a host name. Meaning if you have address 123.456.789.0 on your SMTP server sending emails for smtp.example.com then an appropriate PTR record for that would be smtp.example.com. Seems too simple, but it works since the only one who can really set a PTR record is the owner of the IP address and it can only be set on their hardware. So it acts as a verification point towards who owns/runs/manages that IP address.

• SPF (Sender Policy Framework): Hostname DNS entry level verification. An SPF record—as explained here—is basically a DNS record set by the domain name holder that provides a list of IP addresses and hostnames of servers that are allowed to send emails for that domain name. That again is another verification step that ensures that only the true domain name owner for an SMTP server can send mails. So let’s say a server with the IP address of 123.456.789.9 is sending emails for example.com. We already know that smtp.example.com uses 123.456.789.0, but an SPF record entry for example.com can state, “Hey! 123.456.789.9 is a good server! He’s legit! Respect his emails!”

• DKIM (DomainKeys Identified Mail): Email message level verification. As explained here and on Wikipedia, “DKIM is an email validation system designed to detect email spoofing by providing a mechanism to allow receiving mail exchangers to check that incoming mail from a domain is authorized by that domain’s administrators and that the email (including attachments) has not been modified during transport.” By using cryptographic hashes, DKIM verifies that the mail itself was not filtered or tampered with during transit. This also serves as yet another verification point in the “Are you legit or are you SPAM?” chain.

So in the end, a public-facing SMTP server worth anything will have at least two of these items (PTR and SPF) set to verify that the SMTP server and related email are legit. Not everyone uses DKIM, but it is another layer of validation that is becoming increasingly popular nowadays as SPAMmers become more tenacious in their efforts to send SPAM.

Solution 3:

Most residential ISPs block TCP port 25 (SMTP) in order to prevent you from participating in a spam network. If your PC becomes infected, your PC can start spewing spam at the behest of someone else.

Solution 4:

The other answers are all excellent, and spam does have a lot to do with it.

But there actually is a simpler, more generic, answer: features. Sending email through SMTP is actually a very complex undertaking. Even without spam, you wouldn't want to implement the whole feature set of the SMTP protocol in every email client; you are better off with a dedicated piece of software (sendmail, postfix etc. are the big ones in the *nix world, Exchange in the Windows world).

For example, even at the most basic, a "real" SMTP server has to at least be able to resolve MX records. Then it has to negotiate features (mostly TLS, but there are other features, too). It has to manage queues for retrying, generate non-delivery reports, etc.

And that's just the basic, must-have, functionality without which the server wouldn't even work. It doesn't even include things such as address rewriting, mailertables. Not to mention the dozen or so other protocols that sendmail et al support, such as UUCP.

The SMTP implementation in Outlook, Thunderbird etc. is very minimal - at best, roughly the equivalent of using a smart host on sendmail, if that.

Related, but a separate issue: email is a very security-sensitive topic, and you would want to have one or a very few centrally-managed servers handling it, instead of potentially hundreds or thousands of individual ones on each desktop.

Solution 5:

Why do I need an intermediate SMTP server to send mail? Why can’t my client (Outlook, Thunderbird) send messages directly to the recipient’s SMTP domain?

You could create an email program that did this, and I have no doubt others have done (or attempted) it before too.

You'd essentially be writing a tool that is both a MUA (mail user agent) and MTA (mail transfer agent) in one.

The reason that this is traditionally separated into different tools, with the MTA residing "server-side", is that an MTA that sends mail through the open internet is considerably more complex to write and configure, and also that it benefits from residing on a reliable "always on" server.

An MTA has to:

• Look up and connect to servers it doesn't trust, or which may misbehave, and deal with error conditions in a sensible way that doesn't lose mail.

• Deal with servers that are down, and route to alternate servers or queue the mail for later re-trying. This runs best on a server process that is "always connected" to the internet. It also implies that the mail transfer agent needs its own storage areas for mail that is queued.

• Deal with a range of different server capabilities, adjusting behaviour according to the capabilities of the receiving server.

• Report back to the user about error conditions or when mail is undeliverable, so that mail is not just lost.

• Have excellent security practices and be very security conscious.

• Ideally, reside on a reliable, always-connected server with a stable IP address and a reverse DNS entry, ie an internet connection suitable for public-facing servers. This helps other systems not detect mail sent as spam.

Given these requirements it makes sense to house the SMTP server on a public-facing always-on server somewhere, and to try and use a tool suited to doing that particular job.