SMTP 101: Manual SMTP Sessions

Whether you’re testing a new email service, diagnosing a problem between a client program and SMTP service, looking to write a script to send automated emails, or just wanting to learn more about how SMTP works, this SMTP 101 post is the first in a series designed to help you interact with open, text-based, protocols in common use within the email industry.

But before we get into the how-to, let’s start with the basics…

 

What is SMTP?

Simple Mail Transfer Protocol (SMTP) is “an internet standard for electronic mail (email) transmission. First defined by RFC 821 in 1982, it was updated in 2008 with Extended SMTP additions by RFC 5321, which is the protocol in widespread use today.

Although electronic mail servers and other mail transfer agents use SMTP to send and receive mail messages, user-level client mail applications typically use SMTP only for sending messages to a mail server for relaying. For retrieving messages, client applications usually use either IMAP or POP.

SMTP communication between mail servers uses TCP port 25. Mail clients on the other hand, often submit the outgoing emails to a mail server on port 587 or 465. The latter port was previously deprecated, but this changed with RFC 8314 and its use is now recommended to ensure security. SMTP connections on port 25 or 587 can be secured using an extra command (STARTTLS).

Although proprietary systems… and [some] webmail systems… use their own non-standard protocols to access mail box accounts on their own mail servers, all use SMTP when sending or receiving email from outside their own systems.” Source

SMTP is a plaintext protocol, so you can just type commands from your keyboard and send an email.

In this post’s example, we use the atmail cloud, but it is equally applicable to any SMTP service.

 

What is SMTP? - manual SMTP - atmail email experts

 

Connecting to SMTP

The three options for connecting to SMTP (dependant on your installation) are:

  • plaintext (or Telnet)
  • SSL
  • TLS

Let’s walk through each option…

 

1. Plaintext or Telnet

The easiest method to connect (if your SMTP server supports plaintext connections) is via the program Telnet (short for Terminal Network).  If you don’t have telnet installed and can’t have it installed (like on your locked down production system), you might see if netcat is installed.

In this example, I am using Linux, but you could equally use PuTTY on Windows.

The port will typically be either port 25 or 587.  (Port 465 is typically used for SSL connections.)

Example:

$ telnet smtp-us.atmailcloud.com 25
Trying 204.145.97.24...
Connected to smtp-us.atmailcloud.com.
Escape character is '^]'.
220 us11-011mrc - SMTP ready

Now we can type commands.

Now the same thing with netcat or nc if telnet isn’t available:

$ nc --crlf --verbose smtp-us.atmailcloud.com 25
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 204.145.97.22:25.
220 us11-012mrc - SMTP ready
quit
221 Bye
Ncat: 6 bytes sent, 39 bytes received in 7.90 seconds.

2. SSL

Some people get confused between SSL and TLS.

To make it clear, SSL (Secure Sockets Layer) is used to encrypt the TCP channel before any SMTP protocol is presented to the client.

TLS, on the other hand, encrypts the channel during the SMTP session (i.e. it starts as plaintext but then switches to an encrypted channel).

The easiest way to initiate an SSL encrypted channel is via the OpenSSL s_client command.

Example:

$ openssl s_client -connect smtp-us.atmailcloud.com:465 -crlf -quiet
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
verify return:1
depth=0 CN = *.atmailcloud.com
verify return:1
220 us11-011mrc - SMTP ready

The option “-crlf” just means to issue a carriage return and a line feed on enter and “-quiet” means not to spit out a bunch of details about the encrypted channel.

Feel free to drop “-quiet” to view the output, or if you are keen, add “-debug” for even more detail.

 

3. TLS

As I said above, TLS (Transport Layer Security) starts after connecting with plaintext.

You could start a TLS session (like I demonstrate here) via Telnet, but then you need to negotiate a binary encrypted channel with Telnet.

The ports for TLS are going to be either 25 or 587 and will depend on your installation.

Starting a TLS session over Telnet:

$ telnet smtp-us.atmailcloud.com 25
Trying 204.145.97.22...
Connected to smtp-us.atmailcloud.com.
Escape character is '^]'.
220 us11-010mrc - SMTP ready
EHLO example
250-us11-010mrc.dh.atmailcloud.com Hello example [101.165.155.220]
250-SIZE 52428800
250-8BITMIME
250-PIPELINING
250-AUTH LOGIN PLAIN
250-CHUNKING
250-STARTTLS
250 HELP
STARTTLS
220 TLS go ahead

You can see that I introduced the command “ehlo some-name”.

This just asks the SMTP server which extended options are supported.

One of those returned is ‘STARTTLS’, so I executed that and at that point, the TCP connection switched to binary and I needed to negotiate an encrypted channel.

The easiest way to do this is using OpenSSL s_client again, like so:

$ openssl s_client -connect smtp-us.atmailcloud.com:25 -crlf -quiet -starttls smtp
depth=0 C = UK, O = Exim Developers, CN = us11-011mrc.dh.atmailcloud.com
verify error:num=18:self signed certificate
verify return:1
depth=0 C = UK, O = Exim Developers, CN = us11-011mrc.dh.atmailcloud.com
verify error:num=9:certificate is not yet valid
notBefore=Sep 12 11:29:24 2018 GMT
verify return:1
depth=0 C = UK, O = Exim Developers, CN = us11-011mrc.dh.atmailcloud.com
notBefore=Sep 12 11:29:24 2018 GMT
verify return:1
250 HELP

Now I issue some new commands, for example the ‘EHLO’ we introduced earlier.

250 HELP
EHLO example
250-us11-010mrc.dh.atmailcloud.com Hello example [101.165.155.220]
250-SIZE 52428800
250-8BITMIME
250-PIPELINING
250-AUTH LOGIN PLAIN
250-CHUNKING
250 HELP

You can see, for obvious reasons, that STARTTLS is now missing.

 

Authenticating to SMTP

Sometimes you may want to authenticate to the SMTP server (eg. if it is your outbound mail server).

But you may not always need to authenticate (eg. if you are connecting to an MX server of a particular domain – you won’t be able to authenticate).

But if you do want to authenticate, this is how:

You can see above, in response to the EHLO, we get the line:

250-AUTH LOGIN PLAIN

This means this server supports both LOGIN and PLAIN methods.

 

Authenticating via Login Method

The login method means we send the username and password separately as base64 encoded strings.

SMTP servers tell you this by encoding the question. (Why? Not sure. Probably, just to be clever.)

We issue the command “AUTH LOGIN” and we receive:

AUTH LOGIN
334 VXNlcm5hbWU6

Now, if we decode that base64 string, we get:

$ echo "VXNlcm5hbWU6" | openssl base64 -d
Username:

The same applies for the next prompt, which is “Password:”.

So how do you encode your username and password to send in base64?

This is my preferred way:

$ echo -en "[email protected]" | openssl base64
c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20=
$ echo -en "my^Passw0rd" | base64
bXleUGFzc3cwcmQ=

Now putting this all together to actually authenticate:

EHLO example
250-us11-011mrc.dh.atmailcloud.com Hello o [101.165.155.220]
250-SIZE 52428800
250-8BITMIME
250-PIPELINING
250-AUTH LOGIN PLAIN
250-CHUNKING
250 HELP
AUTH LOGIN
334 VXNlcm5hbWU6
c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20=
334 UGFzc3dvcmQ6
bXleUGFzc3cwcmQ=
235 Authentication succeeded

 

Authenticating via Plain Method

Using the Plain method, we provide the username and password as single base64 encoded string, separated by the NUL character.

Here is how to generate the base64 on a Linux machine:

$ echo -en "[email protected]^Passw0rd" | openssl base64
AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAG15XlBhc3N3MHJk

We will address the leading NUL in the next section, but for basic user authentication, that is the format you need.

Here is an authenticating session:

AUTH PLAIN
334 
AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAG15XlBhc3N3MHJk
235 Authentication succeeded

We are now authenticated and can issue commands as a trusted user.

 

Special Credit: Authenticating via SASL PLAIN as an Admin User

Now, back to that leading NUL in the previous section.  The SASL PLAIN, which is really the Plain method above, defines the ability to authenticate as one user but actually perform the session as if another user authenticated.  The typical use case is for an administrator to authenticate as themselves, but access the service as another user.

We don’t use an admin password for the atmail cloud, but I do have a lab box where I have atmail on-prem running, so I’ll switch to that for the purposes of this section.

The format is:

“authcidNULauthzidNULpassword”

So generate this we can use Linux:

$ echo -en "[email protected]" | openssl base64
dGlnZXJAemV1cy5wAGFkbWluAGFkbWluMTIzNA==

Here

auth plain
334 
dGlnZXJAemV1cy5wAGFkbWluMQBhZG1pbjEyMzQ=
235 Authentication succeeded

 

Sending an Email

Now that we’re connected (possibly via SSL/TLS) and we’ve optionally authenticated, we can send an email.

As far as the SMTP server is concerned, an email consists of three parts:

  1. Who the message is from (MAIL FROM);
  2. Who the message is to (RCPT TO); and
  3. The message (DATA).

At the SMTP layer, the message header and body are just considered the “message” (or more correctly, the “DATA”).

You can only have a single MAIL FROM, but many RCPT TO’s (up to the limit configured by the server administrator).

Note: Some SMTP servers enforce the SMTP protocol more strictly than others. So, while in the examples below I just use the email address form as “[email protected]”, some SMTP servers will insist on the RFC specified format of using brackets (i.e. “<[email protected]>”). Please bear this in mind when you are testing.

MAIL FROM: [email protected]
250 OK
RCPT TO: [email protected]
250 Accepted Recipient
RCPT TO: [email protected]
250 Accepted Recipient
DATA
354 Enter message, ending with "." on a line by itself
From: Some User <[email protected]>
To: Some Other User <[email protected]>
Cc: [email protected]
Subject: Hello

Hello my friends.
.
250 OK id=1g040I-0008DT-GG

So what I did above was:

  1. I issued a MAIL FROM
  2. Then a couple of RCPT TO’s
  3. Then the DATA
  4. I then entered the email as I wished to send
  5. Told the server I was finished by issuing a single ‘.’ on a line by itself.
  6. The server accepted the message.

A couple of points worth noting:

  1. The format of the DATA is up to you, you can include some headers and not others.
  2. The indicator of where message headers stop and the message body begins is a single blank line.
  3. There is no direct relationship between the From and To/CC fields in the DATA and the envelope MAIL FROM and RCPT TO.  Basically, the envelope’s need to be real and exist and the message body can be pure fantasy, this is also known as spoofing (some mail servers will re-write the DATA headers to align with the envelope, but this is not super-common practice).

 

A Complete Example

Here is a complete SMTP session, including some spoofing, as an example.

$ openssl s_client -connect smtp-us.atmailcloud.com:25 -crlf -quiet -starttls smtp
depth=0 C = UK, O = Exim Developers, CN = us11-012mrc.dh.atmailcloud.com
verify error:num=18:self signed certificate
verify return:1
depth=0 C = UK, O = Exim Developers, CN = us11-012mrc.dh.atmailcloud.com
verify return:1
250 HELP
EHLO EXAMPLE
250-us11-012mrc.dh.atmailcloud.com Hello 27-33-184-51.static.tpgi.com.au [27.33.184.51]
250-SIZE 52428800
250-8BITMIME
250-PIPELINING
250-AUTH LOGIN PLAIN
250-CHUNKING
250 HELP
AUTH PLAIN
334 
AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAG15XlBhc3N3MHJk
235 Authentication succeeded
MAIL FROM: [email protected]
250 OK
RCPT TO: [email protected]
250 Accepted Recipient
DATA
354 Enter message, ending with "." on a line by itself
From: [email protected]
To: [email protected]
Subject: Hello everyone!

A little spoofed message from someone pretending to be the [email protected]

I hope you enjoyed this blog post.
.
250 OK id=1g0eWC-0003vH-Su
QUIT
221 Bye

 

Find this Useful?

We’re taking ideas for future email-related 101 posts.

If you’d like to make a request, we invite you to drop us a line here.

 

 

New to atmail?

If email is not your core business and you’d like someone else to worry about SMTP, we can help.

With 20 years of global, white label, email expertise serving telecommunications and hosting providers across every continent, you can trust us to deliver white label, email solutions that are stable, secure and scalable.

We power 170 million mailboxes and offer user-friendly, cloud hosted email with 99.99% uptime and your choice of US or (GDPR compliant) EU data centres.

Or, if you want to stay in-house, we offer on-premises webmail and/or mail server options.

Talk to us today.

 

 

Share This Post
By Dave | September 17, 2018

Leave A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.