How exactly does SMTP authentication work?
I am poking around with SMTP with the hope of getting a better understanding of how it works and I am not sure I understand how authentication works and what controls who can send a message using an email address.
SO basically this is what I have done.
I was able to connect to company's smtp server by running:
telnet smtp.company.com
then I executed these series of commands to send a message to myself:
EHLO company.com
MAIL FROM:<[email protected]>
RCPT TO:<[email protected]>
DATA
Subject: Test
From: [email protected]
To: [email protected]
The content of the message
.
QUIT
I was also able to use a collegues email address to send a message to myself, basically this:
EHLO company.com
MAIL FROM:<[email protected]>
RCPT TO:<[email protected]>
DATA
Subject: Test
From: [email protected]
To: [email protected]
The content of the message
.
QUIT
I was able to do this without knowing or being asked for his password.
Then I also attempted to send to my gmail account, and that failed with the error:
.
421-4.7.0 [MY.LOCAL.IP.ADDRESS 15] Our system has detected that this message is
421-4.7.0 suspicious due to the very low reputation of the sending IP address.
421-4.7.0 To protect our users from spam, mail sent from your IP address has
421-4.7.0 been temporarily rate limited. Please visit
421 4.7.0 https://support.google.com/mail/answer/188131 for more information.
Connection closed by foreign host.
Which I interpret as the only reason this was not allowed was due to the low reputation of my local IP address, and not because of any authentication.
So my question still stands...how does authentication work with email/smtp? How come I could use my colleagues email address to send messages without him authorizing this?
The whole point of authentication is to authorize caller based on authentication results. After authentication, server knows who you are, and can provide you with additionall services. For email, that would be it will relay your mail to outside servers, even it won't relay for not authenticated clients. Note your first two examples weren't destined for relaying, so server accepted mail; that's generally how the conversation between two servers look.
I often configure servers so they also check if authenticated user permitted to use this envelope sender (MAIL FROM) address. So you won't be able to send outside mail with envelope sender set to your boss's email, even if you were authenticated.
There are three ports defined for SMTP. Well known port 25 originally used for all relaying and later for submission from mail user agents (MUA's); now its recommended usage is only relaying (a communication between servers). It is often disabled at ISP level, so users even can not connect to outside port 25, this helps to prevent spam from infected computers. Because MUA's aren't expected on this port, authentication is often disabled on it.
Other port is 587; it is defined as SMTP Submission and it it the port that MUA's clients use. It starts as a plain text, but upgradable to STARTTLS after the command. This enables wider support and some autodetection. This port often uses authentication.
The third port, 465, is SMTPS, is also for submission, but expects clients to initialize a TLS first and then start SMTP session inside it. It also expects MUA's which authenticate themselves.
The SMTP authentication is defined in the RFC 4954, and it is proceeded as follows:
I. It is only supported for ESMTP, i.e. it is an extension; the original SMTP doesn't have authentication. To initialize, you must connect to a server and issue a ESMTP HELLO:
EHLO my.host.name
II. Remote server will show extended reply, which lists extensions it supports at this stage of protocol. We now interested in AUTH and STARTTLS extensions. AUTH may be not available initially (i.e. no line with AUTH), but if there is STARTTLS support, you can issue STARTTLS command and establish a TLS session, then AUTH may appear. Note, you won't be able to use STARTTLS in telnet session; to test how it feels inside of encrypted tunnel just use openssl s_client tool. All in all, if AUTH is in the list, it will be followed with a list of authentication methods are available:
250-AUTH DIGEST-MD5 LOGIN CRAM-MD5 NTLM
III. Then client chooses authentication method and proceeds:
AUTH LOGIN
For LOGIN method server expects two lines, user name and password. It first asks:
334 VXNlcm5hbWU6
If you decode this base64 string, it'll happen this is the word "Username". You answer with username encoded in base64, and it answers with another line:
334 UGFzc3dvcmQ6
This is the word "Password" in base64, and server expects now your plain text password. Again, at this stage you reply with password encoded as base64.
IV. Server will say if authentication is successful or not:
535 5.7.8 Error: authentication failed: authentication failure
235 2.7.0 Authentication successful
After that you continue SMTP session.
Other authentication methods use different procedures. Each of them has some advantages and disadvantages. LOGIN and PLAIN require you to send password in clear text which only should be done over the STARTTLS, but server could store scrambled passwords and no plain-text. With CRAM-MD5 mechanism server presents you with a challenge, and you must calculate a correct response to prove you know a password, without sending it in plain text (or in any reversible encoding) over the wire. It is resistant to eavesdropping over non-encrypted connection, but requires server to store passwords in plain text. DIGEST-MD5 is similar. In general, SRP-6 is thought to be the most secure password-based mechanism, but it hard to implement and it requires quite complicated computations on both sides, so it isn't widely used.
These methods are pluggable. The whole procedure is handled out to SASL library, which implements concrete procedures. In my example SASL library on the server either doesn't support any SRP variant, or it has been disabled in the configuration.
In your case, authentication does not appear to be required, at all.
SMTP auth works the way one configures it. Historically, email software shipped with only formal restrictions on what source and destination addresses even unauthenticated third-parties can use.
Beyond requiring that some sort of authentication (explicit or implicit) should be required before relaying (submitting mail for destinations outside the what the server handles itself), there is not much strong consensus on what to enforce. Most servers will require explicit login before accepting mail presented as originating from a destination they are the final destination for. Most servers will require that the envelope from matches a per-login list of allowed aliases. Some servers will enforce additional restrictions to header from address, possibly even display name.
What level of restrictions is appropriate depends on your environment and what other methods you apply to a) combat and b) diagnose potential abuse.
Recommendations:
- define and check for a list of systems authorized to send on your behalf, and trusted to enforce restrictions: your search keyword on a typical way to do that is "SPF"
- require that all relayed mail can only be submitted after explicit authentication; that means stop providing an "open relay"
How to best do that depends much on the software you use and the mail users (that possible currently depend on the mostly unrestricted configuration) and thus would go beyond what I can put in a single answer, but to make more thing abundantly clear:
I strongly believe that the case you describe: anyone can use any source address in both envelope and headers - is reckless and will be (and possibly already has been) abused to trick your company and others.