POP 101: Manual POP Sessions
If you are needing to test a new email service, diagnose a problem between a client email program and a POP server, wanting to write a script to check for new emails in a mailbox, or just keen to learn more about how POP works, this post (which follows on from SMTP 101: Manual SMTP Sessions as the second in a series of how-to tutorials designed to help you interact with open, text-based protocols in common use within the email industry) is for you.
Let’s start with the basics of POP…
What is POP?
Post Office Protocol (POP) is an internet standard for retrieving electronic mail (email) from a server. RFC 1939 defines the current protocol, which was published in 1996. It has been updated by various errata since then (RFC’s 2449, 5034, 6186 and 8314) – the last of which was in January 2018. So, whilst the protocol is very old, it is still very much in use today.
The current POP standard in use is Post Office Protocol v3, which is interchangeably referred to as POP3, POPv3 or simply, POP. This implies that there was a v1 and v2 of the protocol and this is very true, but you shouldn’t find any of these servers in use today. Or if you do, run!
Just to be very clear, POP is used to retrieve email, you cannot send email via POP. If you want to send email, check out our SMTP post.
POP communication between client and server occurs on TCP port 110 (clear text) or TCP port 995 (SSL). However, many implementations offer and enforce TLS on port 110 (STARTTLS).
POP is a plaintext protocol, so you can just type commands from your keyboard and retrieve an email from your mail server.
In this post’s example, we use the atmail cloud, but it is equally applicable to any POP service.
Connecting to a POP Server
The two main options for connecting to POP (dependant on your installation) are:
- plaintext (or Telnet)
- SSL
Let’s discuss both… and we’ll also touch on TLS.
1. Plaintext or Telnet (or netcat)
The easiest method to connect (if your POP server supports plaintext connections) is via the program Telnet (short for Terminal Network). If you don’t have telnet installed and can’t have it installed (like on your locked down production system), you might see if netcat is installed.
In this example, I am using Linux, but you could equally use PuTTY on Windows.
The port will typically be TCP 110.
Example:
$ telnet pop-us.atmailcloud.com 110 Trying 204.145.97.26... Connected to imap-us.atmailcloud.com. Escape character is '^]'. +OK IMAP/POP3 ready - us11-010mip
Now we can type commands.
The same thing using netcat or nc if telnet isn’t available:
$ nc --crlf --verbose pop-us.atmailcloud.com 110 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 204.145.97.26:110. +OK IMAP/POP3 ready - us11-012mip quit +OK Logging out Ncat: 6 bytes sent, 52 bytes received in 4.50 seconds.
2. SSL
SSL (Secure Sockets Layer) is used to encrypt the TCP channel before any POP protocol is presented to the client. This is the secure encryption standard supported by our atmail cloud.
The easiest way to initiate an SSL encrypted channel is via the OpenSSL s_client command.
Example:
$ openssl s_client -connect pop-us.atmailcloud.com:995 -crlf -quiet depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2 verify return:1 depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1 verify return:1 depth=0 CN = *.atmailcloud.com verify return:1 +OK IMAP/POP3 ready - us11-010mip
The option “-crlf” just means to issue a carriage return and a line feed on enter and “-quiet” means not to spit out a bunch of details about the encrypted channel.
Feel free to drop “-quiet” to view the output, or if you are keen, add “-debug” for even more detail.
A Note on TLS
Some POP installations offer a TLS method of connection, with the STARTTLS or rather, the STLS command. Our atmail cloud does not support this method. Why? Mainly because we, and most of the email industry, discourage the use of POP because, generally speaking, IMAP is a far superior protocol and enables easy device interoperability on the same set of mailboxes.
Short story, you really shouldn’t be using POP in 2018 – it’s probably time to move on.
But if you do want to know how to connect to a POP service that does offer STLS, the easiest way to do this is to use OpenSSL s_client again, like so:
$ openssl s_client -connect server.domain.tld:110 -crlf -quiet -starttls pop3
Authenticating to POP
To actually retrieve email over POP, you need to authenticate. There are three ways to authenticate:
- user/pass (userpass)
- SASL AUTH LOGIN
- SASL AUTH PLAIN
Sometimes you come across the term APOP (or Authenticated POP). APOP is an older method to not transport a user’s password across the internet in plain text, which is achieved by the client pre-hashing the password, using a known and agreed hashing algorithm (MD5).
However, most (if not all) modern implementations have done away with this authentication method, because it typically (but not always) relies on the server storing the user’s password in plaintext somewhere – which is not an ideal service principle.
Instead most modern services rely on SSL to transmit a user’s password over the internet securely (and server-side performs the hash).
Authenticating via the user/pass Method
With this method you use the built-in user and pass commands that are defined in the RFC. It’s quite straight forward:
$ telnet pop-us.atmailcloud.com 110 Trying 204.145.97.27... Connected to imap-us.atmailcloud.com. Escape character is '^]'. +OK IMAP/POP3 ready - us11-010mip user [email protected] +OK pass My_P@ssword1 +OK Logged in.
Now we can issue POP commands.
Authenticating via the SASL LOGIN Method
Firstly, we must establish if the POP server supports the SASL AUTH LOGIN method. We do this by issuing the CAPA command to the POP server, like so:
$ telnet pop-us.atmailcloud.com 110 Trying 204.145.97.26... Connected to imap-us.atmailcloud.com. Escape character is '^]'. +OK IMAP/POP3 ready - us11-010mip CAPA +OK CAPA TOP UIDL RESP-CODES PIPELINING AUTH-RESP-CODE USER SASL PLAIN LOGIN .
So, as you can see here, the POP server running in the atmail cloud supports SASL PLAIN and SASL LOGIN.
The login method means we send the username and password separately as base64 encoded strings.
The POP server tells you this by encoding the question. (Why? Not sure. Probably, just to be clever.)
We issue the command “AUTH LOGIN” and we receive:
AUTH LOGIN + VXNlcm5hbWU6
Now, if we decode that base64 string, we get:
$ echo "VXNlcm5hbWU6" | openssl base64 -d Username:
The same applies for the next prompt, which is “Password:”.
So, how do you encode your username and password to send in base64?
This is my preferred way:
$ echo -en "[email protected]" | openssl base64 c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20= $ echo -en "My_P@ssword1" | base64 TXlfUEBzc3dvcmQx
Now putting this all together to actually authenticate:
+OK IMAP/POP3 ready - us11-012mip AUTH LOGIN + VXNlcm5hbWU6 c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20= + UGFzc3dvcmQ6 TXlfUEBzc3dvcmQx +OK Logged in.
Authenticating via the SASL PLAIN Method
We established above that the POP server supports the SASL AUTH PLAIN method, due to “PLAIN” being listed in the supported SASL types.
Using the PLAIN method, we provide the username and password as single base64 encoded string, separated by the NUL character.
Here is how to generate the base64 on a Linux machine:
$ echo -en "[email protected]_P@ssword1" | openssl base64 AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAE15X1BAc3N3b3JkMQ==
We will address the leading NUL in the next section, but for basic user authentication, that is the format you need.
Here is an authenticated session:
+OK IMAP/POP3 ready - us11-012mip AUTH PLAIN + AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAE15X1BAc3N3b3JkMQ== +OK Logged in.
Special Credit: Authenticating via SASL PLAIN as an Admin User
Now, back to that leading NUL in the previous section. The SASL PLAIN method defines the ability to authenticate as one user, but actually performs the session as if another user authenticated. The typical use case is for an administrator to authenticate as themselves, but access the service as another user.
We don’t use an admin password for atmail cloud, but I do have a lab box where I have atmail on-prem running, so I’ll switch to that for the purposes of this section, because I want to access the mailbox for the user “[email protected]”.
The format is:
“authcidNULauthzidNULpassword”
So, to generate this we can use Linux:
$ echo -en "[email protected]" | openssl base64 dGlnZXJAemV1cy5wAGFkbWluAGFkbWluMTIzNA==
Here is an authenticating session:
+OK IMAP/POP3 ready - zeus auth plain + dGlnZXJAemV1cy5wAGFkbWluAGFkbWluMTIzNA== +OK Logged in.
Retrieving Emails and Modifying the Inbox
Now that we’re connected (possibly via SSL) and we’ve authenticated, we can retrieve emails over POP.
POP is a very simple protocol and the reasons for this are:
- You can only access the inbox for a user;
- You can not set state for a message;
- You can not receive real time updates of a change in a mailbox; and
- Mailbox sharing is not supported.
The commands in common use are:
STAT Command
STAT returns the number of messages in an inbox and their collective size (in octets).
STAT +OK 2 13410
In this case, 2 messages totalling 13,410 octets (or more commonly, bytes).
LIST Command
LIST returns a list of messages in the inbox, their message number and the size in octets (commonly known as bytes).
You may optionally specify a message number and the listing will only return for that message.
All messages:
LIST +OK 2 messages: 1 8623 2 4787 .
Only a single message:
LIST 2 +OK 2 4787
RETR Command
RETR retrieves a specified message. You need to specify a message number from the LIST command. This will return the full RFC 822 message. In the OK banner will be the size of the message.
<snip>’ed areas are just truncation by the author to remove unnecessary information.
RETR 2 +OK 4787 octets Return-Path: <[email protected]> Delivered-To: [email protected] <snip> Content-Type: text/plain; charset=us-ascii Subject: Test message 1 Date: Wed, 3 Oct 2018 11:01:29 +1000 To: [email protected] This is just a test message for the POP blog post for atmail. .
DELE Command
DELE will delete a message from the server, which clients will typically do once they have downloaded the message from the server.
LIST +OK 2 messages: 1 8623 2 4787 . DELE 2 +OK Marked to be deleted. LIST +OK 1 messages: 1 8623 .
Here I see that two messages were in the inbox.
I deleted message number two and now there is only one message in the inbox.
To be 100 percent technically correct, the actual delete will not happen until the client executes the “QUIT” command, so if this command is never executed (and rather the client just terminates the TCP connection), you will find the message in the inbox next time you login.
NOOP Command
The NOOP command literally does no operation, nothing, nada, zilch, it just returns OK.
It’s useful for a client to keep a connection open, for a reason only known to it.
NOOP +OK
RSET Command
RSET instructs the server to unmark any messages that have been marked for deletion during this session (but not from previous sessions).
LIST +OK 2 messages: 1 8623 2 4787 . DELE 2 +OK Marked to be deleted. LIST +OK 1 messages: 1 8623 . RSET +OK LIST +OK 2 messages: 1 8623 2 4787
Here we see that there are two messages in the inbox.
We delete message number two, verify it no longer appears in the listing, then perform a reset (RSET) and the message re-appears in the listing.
QUIT Command
QUIT tells the server to please commit all changes the client requested and terminate the session.
This is the last command to be issued by a client.
QUIT +OK Logging out.
Extra Commands
There are two extra commands that are in common use in the POP world, so we will quickly discuss them too.
UIDL Command
On each successive login to a POP server, the message number might change due to manipulations made in previous sessions. For example, a server may report that eight messages are in an inbox and the client subsequently deletes messages two, five and seven, leaving five messages on the server. The next session will report messages one to five available. As such, the client cannot easily identify messages that it has already downloaded without downloading the entire contents and comparing it to messages it already has – which is a rather pointless activity.
UIDL to the rescue, known as “UID Listing”, which is a replacement command for LIST. The server assigns a unique identifier to every message on the server and will only ever report that message as having a particular UIDL. The client can rely on this identifier and accordingly make a quick assessment as to if the client has ever downloaded that message. On issuing this command, a listing is provided with both a message number and a unique identifier (UID). All subsequent commands will still require the message number though.
Here is a quick example:
LIST +OK 2 messages: 1 8623 2 4787 . UIDL +OK 1 000000015b98f419 2 000000045b98f419 . UIDL 2 +OK 2 000000045b98f419
As you can see, I may optionally specify a message number if I want the UIDL of a particular message, or simply retrieve the UIDL for all messages.
Now have a look at what happens when I delete message number one (UIDL 000000015b98f419):
Session 1:
LIST
+OK 2 messages:
1 8623
2 4787
.
UIDL
+OK
1 000000015b98f419
2 000000045b98f419
.
DELE 1
+OK Marked to be deleted.
QUIT
+OK Logging out, messages deleted.
Session 2:
LIST +OK 1 messages: 1 4787 . UIDL +OK 1 000000045b98f419 .
As you can see the message number changes in session 2, but the UIDL remains constant.
TOP Command
The TOP command allows partial message retrieval for clients and protects against the bandwidth requirements of downloading the full message (for whatever reason it may wish to do so – perhaps message previews).
When issued, the full header contents of a message are retrieved, as well as the top “n” lines, where “n” is specified in the command.
In this example I ask for only the first five lines of message number one and I receive the headers for free. I have trimmed these header lines to make it easier to read:
TOP 1 5 +OK Return-Path: <[email protected]> Delivered-To: [email protected] <snip> Content-Type: text/plain; charset=us-ascii Subject: Test message 2 Date: Wed, 3 Oct 2018 11:08:34 +1000 To: [email protected] This is just a test message for the POP blog post for atmail, it contains 15 lines, but you will only see the top five. I bet you are wondering what else are on the other lines? Well, that is certainly an interesting question, all I can say is that .
C’mon, you have to the leave the reader hanging…
A Complete Example
Here is a complete POP session utilising the atmail cloud, as an example.
$ openssl s_client -connect pop-us.atmailcloud.com:995 -crlf -quiet depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2 verify return:1 depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1 verify return:1 depth=0 CN = *.atmailcloud.com verify return:1 +OK IMAP/POP3 ready - us11-012mip auth plain + AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAE15X1BAc3N3b3JkMQ== +OK Logged in. LIST +OK 5 messages: 1 4787 2 4664 3 4444 4 4111 5 3621 . LIST 4 +OK 4 4111 UIDL +OK 1 000000045b98f419 2 000000055b98f419 3 000000065b98f419 4 000000075b98f419 5 000000085b98f419 . UIDL 2 +OK 2 000000055b98f419 RETR 5 +OK 3621 octets Return-Path: <[email protected]> Delivered-To: [email protected] Received: from us11-011ms.dh.atmailcloud.com by us11-011ms.dh.atmailcloud.com (Dovecot) with LMTP id KmmTGH0mtFtoSQAAsct0AA for <[email protected]>; Wed, 03 Oct 2018 12:18:16 +1000 Received: from us11-012mrr.dh.atmailcloud.com ([10.10.5.22]) by us11-011ms.dh.atmailcloud.com with esmtp (Exim 4.90_1) (envelope-from <[email protected]>) id 1g7Wjv-0004vJ-VN for [email protected]; Wed, 03 Oct 2018 12:18:15 +1000 Received: from us11-012mx.dh.atmailcloud.com ([10.10.4.22]) by us11-012mrr.dh.atmailcloud.com with esmtp (Exim 4.90_1) (envelope-from <[email protected]>) id 1g7Wgg-0005AY-1M for [email protected]; Wed, 03 Oct 2018 12:14:54 +1000 Received: from nsstlmta32p.somedomain.com ([172.15.1.1]) by us11-012mx.dh.atmailcloud.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.90_1) (envelope-from <[email protected]>) id 1g7Wi4-0007bH-20 for [email protected]; Wed, 03 Oct 2018 12:16:20 +1000 Received: from smtp.somedomain.com ([10.10.24.4]) by nsstlfep32p-svc.somedomain.com with ESMTP id <[email protected]> for <[email protected]>; Wed, 3 Oct 2018 12:13:58 +1000 From: The Author <[email protected]> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 11.5 (3445.9.1)) Subject: Test Message 6 Message-Id: <[email protected]> Date: Wed, 3 Oct 2018 12:13:56 +1000 To: [email protected] X-Mailer: Apple Mail (2.3445.9.1) Just another test message . DELE 5 +OK Marked to be deleted. NOOP +OK LIST +OK 4 messages: 1 4787 2 4664 3 4444 4 4111 . RSET +OK LIST +OK 5 messages: 1 4787 2 4664 3 4444 4 4111 5 3621 . DELE 3 +OK Marked to be deleted. QUIT +OK Logging out, messages deleted.
Find this Useful?
We’re taking ideas for future email-related posts.
If you’d like to make a request, we invite you to drop us a line here.
New to atmail?
If email is not your core business and you’d like someone else to worry about POP, we can help.
With 20 years of global, white label, email expertise serving telecommunications and hosting providers across every continent, you can trust us to deliver white label, email solutions that are stable, secure and scalable.
We power 170 million mailboxes and offer user-friendly, cloud hosted email with 99.99% uptime and your choice of US or (GDPR compliant) EU data centres.
Or, if you want to stay in-house, we offer on-premises webmail and/or mail server options.
More Posts
