SMTP 101: Manual SMTP Sessions
Whether you’re testing a new email service, diagnosing a problem between a client program and SMTP service, looking to write a script to send automated emails, or just wanting to learn more about how SMTP works, this SMTP 101 post is the first in a series designed to help you interact with open, text-based, protocols in common use within the email industry.
But before we get into the how-to, let’s start with the basics…
What is SMTP?
Simple Mail Transfer Protocol (SMTP) is “an internet standard for electronic mail (email) transmission. First defined by RFC 821 in 1982, it was updated in 2008 with Extended SMTP additions by RFC 5321, which is the protocol in widespread use today.
Although electronic mail servers and other mail transfer agents use SMTP to send and receive mail messages, user-level client mail applications typically use SMTP only for sending messages to a mail server for relaying. For retrieving messages, client applications usually use either IMAP or POP.
SMTP communication between mail servers uses TCP port 25. Mail clients on the other hand, often submit the outgoing emails to a mail server on port 587 or 465. The latter port was previously deprecated, but this changed with RFC 8314 and its use is now recommended to ensure security. SMTP connections on port 25 or 587 can be secured using an extra command (STARTTLS).
Although proprietary systems… and [some] webmail systems… use their own non-standard protocols to access mail box accounts on their own mail servers, all use SMTP when sending or receiving email from outside their own systems.” Source
SMTP is a plaintext protocol, so you can just type commands from your keyboard and send an email.
In this post’s example, we use the atmail cloud, but it is equally applicable to any SMTP service.
Connecting to SMTP
The three options for connecting to SMTP (dependant on your installation) are:
- plaintext (or Telnet)
- SSL
- TLS
Let’s walk through each option…
1. Plaintext or Telnet
The easiest method to connect (if your SMTP server supports plaintext connections) is via the program Telnet (short for Terminal Network). If you don’t have telnet installed and can’t have it installed (like on your locked down production system), you might see if netcat is installed.
In this example, I am using Linux, but you could equally use PuTTY on Windows.
The port will typically be either port 25 or 587. (Port 465 is typically used for SSL connections.)
Example:
$ telnet smtp-us.atmailcloud.com 25 Trying 204.145.97.24... Connected to smtp-us.atmailcloud.com. Escape character is '^]'. 220 us11-011mrc - SMTP ready
Now we can type commands.
Now the same thing with netcat or nc if telnet isn’t available:
$ nc --crlf --verbose smtp-us.atmailcloud.com 25 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 204.145.97.22:25. 220 us11-012mrc - SMTP ready quit 221 Bye Ncat: 6 bytes sent, 39 bytes received in 7.90 seconds.
2. SSL
Some people get confused between SSL and TLS.
To make it clear, SSL (Secure Sockets Layer) is used to encrypt the TCP channel before any SMTP protocol is presented to the client.
TLS, on the other hand, encrypts the channel during the SMTP session (i.e. it starts as plaintext but then switches to an encrypted channel).
The easiest way to initiate an SSL encrypted channel is via the OpenSSL s_client command.
Example:
$ openssl s_client -connect smtp-us.atmailcloud.com:465 -crlf -quiet depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2 verify return:1 depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1 verify return:1 depth=0 CN = *.atmailcloud.com verify return:1 220 us11-011mrc - SMTP ready
The option “-crlf” just means to issue a carriage return and a line feed on enter and “-quiet” means not to spit out a bunch of details about the encrypted channel.
Feel free to drop “-quiet” to view the output, or if you are keen, add “-debug” for even more detail.
3. TLS
As I said above, TLS (Transport Layer Security) starts after connecting with plaintext.
You could start a TLS session (like I demonstrate here) via Telnet, but then you need to negotiate a binary encrypted channel with Telnet.
The ports for TLS are going to be either 25 or 587 and will depend on your installation.
Starting a TLS session over Telnet:
$ telnet smtp-us.atmailcloud.com 25 Trying 204.145.97.22... Connected to smtp-us.atmailcloud.com. Escape character is '^]'. 220 us11-010mrc - SMTP ready EHLO example 250-us11-010mrc.dh.atmailcloud.com Hello example [101.165.155.220] 250-SIZE 52428800 250-8BITMIME 250-PIPELINING 250-AUTH LOGIN PLAIN 250-CHUNKING 250-STARTTLS 250 HELP STARTTLS 220 TLS go ahead
You can see that I introduced the command “ehlo some-name”.
This just asks the SMTP server which extended options are supported.
One of those returned is ‘STARTTLS’, so I executed that and at that point, the TCP connection switched to binary and I needed to negotiate an encrypted channel.
The easiest way to do this is using OpenSSL s_client again, like so:
$ openssl s_client -connect smtp-us.atmailcloud.com:25 -crlf -quiet -starttls smtp depth=0 C = UK, O = Exim Developers, CN = us11-011mrc.dh.atmailcloud.com verify error:num=18:self signed certificate verify return:1 depth=0 C = UK, O = Exim Developers, CN = us11-011mrc.dh.atmailcloud.com verify error:num=9:certificate is not yet valid notBefore=Sep 12 11:29:24 2018 GMT verify return:1 depth=0 C = UK, O = Exim Developers, CN = us11-011mrc.dh.atmailcloud.com notBefore=Sep 12 11:29:24 2018 GMT verify return:1 250 HELP
Now I issue some new commands, for example the ‘EHLO’ we introduced earlier.
250 HELP EHLO example 250-us11-010mrc.dh.atmailcloud.com Hello example [101.165.155.220] 250-SIZE 52428800 250-8BITMIME 250-PIPELINING 250-AUTH LOGIN PLAIN 250-CHUNKING 250 HELP
You can see, for obvious reasons, that STARTTLS is now missing.
Authenticating to SMTP
Sometimes you may want to authenticate to the SMTP server (eg. if it is your outbound mail server).
But you may not always need to authenticate (eg. if you are connecting to an MX server of a particular domain – you won’t be able to authenticate).
But if you do want to authenticate, this is how:
You can see above, in response to the EHLO, we get the line:
250-AUTH LOGIN PLAIN
This means this server supports both LOGIN and PLAIN methods.
Authenticating via Login Method
The login method means we send the username and password separately as base64 encoded strings.
SMTP servers tell you this by encoding the question. (Why? Not sure. Probably, just to be clever.)
We issue the command “AUTH LOGIN” and we receive:
AUTH LOGIN 334 VXNlcm5hbWU6
Now, if we decode that base64 string, we get:
$ echo "VXNlcm5hbWU6" | openssl base64 -d Username:
The same applies for the next prompt, which is “Password:”.
So how do you encode your username and password to send in base64?
This is my preferred way:
$ echo -en "[email protected]" | openssl base64 c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20= $ echo -en "my^Passw0rd" | base64 bXleUGFzc3cwcmQ=
Now putting this all together to actually authenticate:
EHLO example 250-us11-011mrc.dh.atmailcloud.com Hello o [101.165.155.220] 250-SIZE 52428800 250-8BITMIME 250-PIPELINING 250-AUTH LOGIN PLAIN 250-CHUNKING 250 HELP AUTH LOGIN 334 VXNlcm5hbWU6 c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20= 334 UGFzc3dvcmQ6 bXleUGFzc3cwcmQ= 235 Authentication succeeded
Authenticating via Plain Method
Using the Plain method, we provide the username and password as single base64 encoded string, separated by the NUL character.
Here is how to generate the base64 on a Linux machine:
$ echo -en '\[email protected]\0my^Passw0rd' | openssl base64 AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAG15XlBhc3N3MHJk
We will address the leading NUL in the next section, but for basic user authentication, that is the format you need.
Here is an authenticating session:
AUTH PLAIN
334
AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAG15XlBhc3N3MHJk
235 Authentication succeeded
We are now authenticated and can issue commands as a trusted user.
Special Credit: Authenticating via SASL PLAIN as an Admin User
Now, back to that leading NUL in the previous section. The SASL PLAIN, which is really the Plain method above, defines the ability to authenticate as one user but actually perform the session as if another user authenticated. The typical use case is for an administrator to authenticate as themselves, but access the service as another user.
We don’t use an admin password for the atmail cloud, but I do have a lab box where I have atmail on-prem running, so I’ll switch to that for the purposes of this section.
The format is:
“authcidNULauthzidNULpassword”
So generate this we can use Linux:
$ echo -en '\[email protected]\0admin1\0admin1234' | openssl base64 dGlnZXJAemV1cy5wAGFkbWluAGFkbWluMTIzNA==
Here
auth plain
334
dGlnZXJAemV1cy5wAGFkbWluMQBhZG1pbjEyMzQ=
235 Authentication succeeded
Sending an Email
Now that we’re connected (possibly via SSL/TLS) and we’ve optionally authenticated, we can send an email.
As far as the SMTP server is concerned, an email consists of three parts:
- Who the message is from (MAIL FROM);
- Who the message is to (RCPT TO); and
- The message (DATA).
At the SMTP layer, the message header and body are just considered the “message” (or more correctly, the “DATA”).
You can only have a single MAIL FROM, but many RCPT TO’s (up to the limit configured by the server administrator).
Note: Some SMTP servers enforce the SMTP protocol more strictly than others. So, while in the examples below I just use the email address form as “[email protected]”, some SMTP servers will insist on the RFC specified format of using brackets (i.e. “<[email protected]>”). Please bear this in mind when you are testing.
MAIL FROM: [email protected] 250 OK RCPT TO: [email protected] 250 Accepted Recipient RCPT TO: [email protected] 250 Accepted Recipient DATA 354 Enter message, ending with "." on a line by itself From: Some User <[email protected]> To: Some Other User <[email protected]> Cc: [email protected] Subject: Hello Hello my friends. . 250 OK id=1g040I-0008DT-GG
So what I did above was:
- I issued a MAIL FROM
- Then a couple of RCPT TO’s
- Then the DATA
- I then entered the email as I wished to send
- Told the server I was finished by issuing a single ‘.’ on a line by itself.
- The server accepted the message.
A couple of points worth noting:
- The format of the DATA is up to you, you can include some headers and not others.
- The indicator of where message headers stop and the message body begins is a single blank line.
- There is no direct relationship between the From and To/CC fields in the DATA and the envelope MAIL FROM and RCPT TO. Basically, the envelope’s need to be real and exist and the message body can be pure fantasy, this is also known as spoofing (some mail servers will re-write the DATA headers to align with the envelope, but this is not super-common practice).
A Complete Example
Here is a complete SMTP session, including some spoofing, as an example.
$ openssl s_client -connect smtp-us.atmailcloud.com:25 -crlf -quiet -starttls smtp depth=0 C = UK, O = Exim Developers, CN = us11-012mrc.dh.atmailcloud.com verify error:num=18:self signed certificate verify return:1 depth=0 C = UK, O = Exim Developers, CN = us11-012mrc.dh.atmailcloud.com verify return:1 250 HELP EHLO EXAMPLE 250-us11-012mrc.dh.atmailcloud.com Hello 27-33-184-51.static.tpgi.com.au [27.33.184.51] 250-SIZE 52428800 250-8BITMIME 250-PIPELINING 250-AUTH LOGIN PLAIN 250-CHUNKING 250 HELP AUTH PLAIN 334 AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAG15XlBhc3N3MHJk 235 Authentication succeeded MAIL FROM: [email protected] 250 OK RCPT TO: [email protected] 250 Accepted Recipient DATA 354 Enter message, ending with "." on a line by itself From: [email protected] To: [email protected] Subject: Hello everyone! A little spoofed message from someone pretending to be the [email protected] I hope you enjoyed this blog post. . 250 OK id=1g0eWC-0003vH-Su QUIT 221 Bye
Find this SMTP post useful?
We’re taking ideas for future email-related 101 posts. If you’d like to make a request, we invite you to drop us a line here.
Meanwhile, we invite you to check out some of our other how-to posts:
- IMAP 101: Manual IMAP Sessions
- IMAP Commands
- Advanced IMAP
- POP 101: Manual POP Sessions
- SMTP Reply, Status and Error Codes
- CalDAV and CardDAV
New to atmail?
If email is not your core business and you’d like someone else to worry about SMTP, we can help.
With 20 years of global, white label, email expertise serving telecommunications and hosting providers across every continent, you can trust us to deliver white label, email solutions that are stable, secure and scalable.
We power 170 million mailboxes and offer user-friendly, cloud hosted email with 99.99% uptime and your choice of US or (GDPR compliant) EU data centres.
Or, if you want to stay in-house, we offer on-premises webmail and/or mail server options.
More Posts
