If you are wanting to test a new email service, diagnose a problem between a client email program and an IMAP server, wanting to write a script to check for new emails in a mailbox, or just keen to learn more about how IMAP works, this post is for you. It will cover: IMAP basics; logging in; selecting the Inbox; retrieving messages; deleting messages; and emptying the Inbox of deleted messages. It follows on from our POP and SMTP posts as the third article in a series of how-to tutorials designed to help you interact with open, text-based protocols in common use within the email industry.

Let’s get started…

What is IMAP?

Internet Message Access Protocol (IMAP) is an internet standard for retrieving electronic mail (email) from a server. RFC 3501 defines the current protocol, which was published in 2003. It has been updated by various errata since then (see RFC 3501) – the last of which was in September 2018. So, whilst the protocol is mature, it is continuing to be enhanced.

The current IMAP standard in use is Internet Message Access Protocol v4rev1, which is interchangeably referred to as IMAP4, IMAPv4rev1 or simply, IMAP.

Just to be very clear, IMAP is used to retrieve email. You cannot send email via IMAP. If you want to send email, please check out our SMTP post.

IMAP communication between client and server occurs on TCP port 143 (clear text) or TCP port 993 (SSL). However, many implementations offer and enforce TLS on port 143 (STARTTLS).

IMAP is a plaintext protocol, so you can just type commands from your keyboard and retrieve an email from your mail server.

In this post’s example, we use the atmail cloud, but it is equally applicable to any IMAP service.

Connecting to an IMAP Server

The two main options for connecting to IMAP (dependant on your installation) are:

plaintext (or Telnet)
SSL

Let’s discuss both… and we’ll also touch on TLS.

A Quick Note on Tags

All IMAP commands are preceded with a command tag, which is just a reference that you, as the client, can make up. It can be something simple, such as “a”, or more complex, such as “TRANS00912.1”. This is provided so that it is clear to which command the IMAP server is responding to by referencing this tag in its response. So, in theory, you can issue many requests without having to wait for each command to return a result. Then, when the result is returned, it is marked as a response to a particular tag.

We will be using tags in these examples here, as it is a mandatory part of the protocol, but we won’t be issuing asynchronous commands.

1. Plaintext or Telnet (or Netcat)

The easiest method to connect (if your IMAP server supports plaintext connections) is via the program Telnet (short for Terminal Network). If you don’t have Telnet installed and can’t have it installed (eg. on your locked-down production system), you might see if Netcat is installed.

In this example, I am using Linux, but you could equally use PuTTY on Windows.

The port will typically be TCP 143.

Example:

$ telnet imap-us.atmailcloud.com 143
Trying 204.145.97.25...
Connected to imap-us.atmailcloud.com.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-012mip

Now we can type commands.

The same thing using netcat or nc if telnet isn’t available:

$ nc --crlf --verbose imap-us.atmailcloud.com 143
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 204.145.97.26:143.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-012mip

2. SSL

SSL (Secure Sockets Layer) is used to encrypt the TCP channel before any IMAP protocol is presented to the client. This is the secure encryption standard supported by our atmail cloud.

The easiest way to initiate an SSL encrypted channel is via the OpenSSL s_client command.

Example:

$ openssl s_client -connect imap-us.atmailcloud.com:993 -crlf -quiet
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
verify return:1
depth=0 CN = *.atmailcloud.com
verify return:1
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-010mip

The option “-crlf” just means to issue a carriage return and a line feed on enter and “-quiet” means not to spit out a bunch of details about the encrypted channel.

Feel free to drop “-quiet” to view the output, or if you are keen, add “-debug” for even more detail.

A Note on TLS

Some IMAP installations offer a TLS method of connection, with the STARTTLS command, on the cleartext port 143. Our atmail cloud does not support this method but we do support the implicit TLS option of using port 993.

But if you do want to know how to connect to an IMAP service that does offer STARTTLS, the easiest way to do this is to use OpenSSL s_client again, as follows:

$ openssl s_client -connect server.domain.tld:143 -crlf -quiet -starttls imap

Authenticating to IMAP

To actually perform actions against an IMAP server, you typically need to authenticate. There are three ways to authenticate:

login
SASL AUTH LOGIN
SASL AUTH PLAIN

Authenticating via Login Method

With this method, you use the built-in login commands that are defined in the RFC. It’s quite straight forward:

$ telnet imap-us.atmailcloud.com 143
Trying 204.145.97.26...
Connected to imap-us.atmailcloud.com.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-011mip
A1 login [email protected] My_P@ssword1
A1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY SPECIAL-USE QUOTA] Logged in

Now we can issue IMAP commands.

Authenticating via SASL LOGIN Method

Firstly, we must establish if the IMAP server supports the SASL AUTH LOGIN method. We can see this in the welcome banner, as follows:

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-011mip

The “AUTH=LOGIN” flag tells us that it supports the SASL LOGIN method, which means we send the username and password separately as base64 encoded strings.

The IMAP server tells you this by encoding the question. (Why does it do this? Not sure. Probably, just to be clever.)

We issue the command “AUTHENTICATE LOGIN” and we receive:

a AUTHENTICATE LOGIN
+ VXNlcm5hbWU6

Now, if we decode that base64 string, we get:

$ echo "VXNlcm5hbWU6" | openssl base64 -d
Username:

The same applies for the next prompt, which is “Password:”.

So, how do you encode your username and password to send in base64?

This is my preferred way:

$ echo -en "[email protected]" | openssl base64
c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20=
$ echo -en "My_P@ssword1" | openssl base64
TXlfUEBzc3dvcmQx

Now, putting this all together to actually authenticate:

$ telnet imap-us.atmailcloud.com 143
Trying 204.145.97.25...
Connected to imap-us.atmailcloud.com.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-012mip
a authenticate login
+ VXNlcm5hbWU6
c29tZXVzZXJAZXhhbXBsZS5hdG1haWxjbG91ZC5jb20=
+ UGFzc3dvcmQ6
TXlfUEBzc3dvcmQx
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY SPECIAL-USE QUOTA] Logged in

Authenticating via SASL PLAIN Method

We established above that the IMAP server supports the SASL AUTH PLAIN method, due to “AUTH=PLAIN” being listed in the supported SASL types.

Using the PLAIN method, we provide the username and password as a single base64 encoded string, separated by the NUL character.

Here is how to generate the base64 on a Linux machine:

$ echo -en "\[email protected]\0My_P@ssword1" | openssl base64
AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAE15X1BAc3N3b3JkMQ==

We will address the leading NUL in the next section, but for basic user authentication, this is the format that you need.

Here is an authenticated session:

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - us11-011mip
a authenticate plain
+ 
AHNvbWV1c2VyQGV4YW1wbGUuYXRtYWlsY2xvdWQuY29tAE15X1BAc3N3b3JkMQ==
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY SPECIAL-USE QUOTA] Logged in

Special Credit: Authenticating via SASL PLAIN as an Admin User

Now, back to that leading NUL in the previous section. The SASL PLAIN method defines the ability to authenticate as one user, but actually performs the session as if another user authenticated. The typical use case is for an administrator to authenticate as themselves, but access the service as another user.

We don’t use an admin password for atmail cloud, but I do have a lab box where I have atmail on-prem running, so I’ll switch to that for the purposes of this section, because I want to access the mailbox for the user “[email protected]”.

The format is:

“authcidNULauthzidNULpassword”

So, to generate this we can use Linux:

$ echo -en "[email protected]" | openssl base64
dGlnZXJAemV1cy5wAGFkbWluAGFkbWluMTIzNA==

Here is an authenticating session:

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] IMAP/POP3 ready - zeus
a authenticate plain
+ 
dGlnZXJAemV1cy5wAGFkbWluAGFkbWluMTIzNA==
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY SPECIAL-USE QUOTA] Logged in

Retrieving Emails and Modifying the Inbox

Now that we’re connected (possibly via SSL) and we’ve authenticated, we can start manipulating messages stored on the server.

IMAP is a complex protocol and supports many, many commands for both manipulating folders and messages (including adding messages to folders, deleting messages, moving, copying and even searching). In this example, we are simply going to select the Inbox, check how many messages are there, retrieve a couple of unread messages and delete a message.

In a follow up post, we will describe the more complex usages.

Namespace

The namespace should be the first command issued, as it will return the prefix and hierarchy delimiter to the Personal Namespace(s), Other Users’ Namespace(s) and Shared Namespace(s) that the server wishes to expose.

We will only be operating in the Personal Namespace in this example, i.e. the users folders and messages.

n namespace
* NAMESPACE (("INBOX/" "/")("virtual/" "/")) NIL NIL
n OK Namespace completed.

Here you can see two personal namespaces and no shared or public namespaces. The folder separator is a “/” character. Our folders are stored under “INBOX/”.

Select the Inbox

Now we have to select a folder to perform functions on. Let’s first get a list of folders:

A1 list "INBOX/" "*"
* LIST (HasNoChildren) "/" INBOX/some_other_folder
* LIST (HasNoChildren UnMarked Archive) "/" INBOX/Archive
* LIST (HasNoChildren UnMarked Sent) "/" INBOX/Sent
* LIST (HasNoChildren Marked Trash) "/" INBOX/Trash
* LIST (HasNoChildren Marked Junk) "/" INBOX/Spam
* LIST (HasNoChildren UnMarked Drafts) "/" INBOX/Drafts
A1 OK List completed (0.000 + 0.000 secs).

So, in addition to the INBOX (which is a special folder that implicitly exists for a user), we can see a bunch of other folders. You can see some are tagged with flags, for example “Sent”. These tags tell the client what type of mailbox this is. In the case of Sent, it means you should use this folder to store Sent messages in.

Let’s select the Inbox:

g21 SELECT "INBOX"
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft *)] Flags permitted.
* 4 EXISTS
* 0 RECENT
* OK [UNSEEN 2] First unseen.
* OK [UIDVALIDITY 1536750617] UIDs valid
* OK [UIDNEXT 9] Predicted next UID
* OK [HIGHESTMODSEQ 11] Highest
g21 OK [READ-WRITE] Select completed (0.000 + 0.000 secs).

We can see here that there are four messages (with message number 2 being the first of which that does not have the Seen flag set).

Fetch Messages

When you fetch messages from IMAP you can fetch the whole message, or message parts, or certain headers, or some meta information like the date the message was stored on the server. You can retrieve a single message or a subset of messages. Let’s get the Subject for all messages in the Inbox:

f fetch 1:4 (BODY[HEADER.FIELDS (Subject)])
* 1 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {27}
Subject: Test message 1

)
* 2 FETCH (FLAGS (Seen) BODY[HEADER.FIELDS (SUBJECT)] {27}
Subject: Test message 3

)
* 3 FETCH (FLAGS (Seen) BODY[HEADER.FIELDS (SUBJECT)] {27}
Subject: Test message 5

)
* 4 FETCH (FLAGS (Seen) BODY[HEADER.FIELDS (SUBJECT)] {27}
Subject: Test Message 6

)
f OK Fetch completed (0.002 + 0.000 secs).

Now let’s find out which messages have yet to be “seen” by any client. We do this by issuing a search:

s search UNSEEN
* SEARCH 2
s OK Search completed (0.001 + 0.000 secs).

In this case, a new message, message 2, is the only unseen message. Let’s retrieve the entire message:

F1 fetch 2 RFC822
* 2 FETCH (FLAGS (Seen) RFC822 {3009}
Return-Path: <[email protected]>
Delivered-To: [email protected]
Received: from us11-011ms.dh.atmailcloud.com
	by us11-011ms.dh.atmailcloud.com (Dovecot) with LMTP id beGRJdw7F1xXTgAAsct0AA
	for <[email protected]>; Mon, 17 Dec 2018 16:02:32 +1000
Received: from us11-010mrr.dh.atmailcloud.com ([10.10.5.20])
	by us11-011ms.dh.atmailcloud.com with esmtp (Exim 4.90_1)
	(envelope-from <[email protected]>)
	id 1gYlz6-0005Df-Hx
	for [email protected]; Mon, 17 Dec 2018 16:02:32 +1000
Received: from us11-002mrs.dh.atmailcloud.com ([10.10.10.11])
	by us11-010mrr.dh.atmailcloud.com with esmtp (Exim 4.90_1)
	(envelope-from <[email protected]>)
	id 1gYlsn-00047x-0z
	for [email protected]; Mon, 17 Dec 2018 15:56:01 +1000
Received: from us11-011mrc.dh.atmailcloud.com ([10.10.3.21])
	by us11-002mrs.dh.atmailcloud.com with esmtp (Exim 4.84)
	(envelope-from <[email protected]>)
	id 1gYlvd-0007o7-7L
	for [email protected]; Mon, 17 Dec 2018 15:58:57 +1000
Received: from us11-011wui.dh.atmailcloud.com ([10.10.1.21] helo=localhost)
	by us11-011mrc.dh.atmailcloud.com with esmtpa (Exim 4.90_1)
	(envelope-from <[email protected]>)
	id 1gYluP-0005tV-1x
	for [email protected]; Mon, 17 Dec 2018 15:57:41 +1000
To: <[email protected]>
Mime-Version: 1.0
Reply-To: "Some User" <[email protected]>
X-Mailer: atmail api 8.4.1-202
Message-Id: <cb6b04b3-b27b-41bb-b1b8-adc66e89fae6@localhost>
Date: Mon, 17 Dec 2018 15:57:12 +1000
Content-Type: multipart/alternative; boundary=5494146fa3cf177fb921d2cc0b347cd9cafaf36b1d18a797c5469f95440b
From: "Some User" <[email protected]>
Subject: Test message 3
X-Atmail-Id: [email protected]
X-atmail-spam-score: 0 
X-atmail-spam-bar: / 

--5494146fa3cf177fb921d2cc0b347cd9cafaf36b1d18a797c5469f95440b
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

t3
--5494146fa3cf177fb921d2cc0b347cd9cafaf36b1d18a797c5469f95440b
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8

<div>t3</div><div><br></div><div data-atmail-signature=3D"" class=3D"gmail_=
signature" data-smartmail=3D"gmail_signature" style=3D""><br></div><div><br=
></div>
--5494146fa3cf177fb921d2cc0b347cd9cafaf36b1d18a797c5469f95440b--
)
F1 OK Fetch completed (0.002 + 0.000 secs).

Delete a Message

A deletion of a message is a two-step process. First, you must store the Delete flag on a message. In our case, against message number 2. Then you must expunge the folder, which has the effect of deleting all messages with the Deleted flag set.

d store 2 +FLAGS (\Deleted)
* 2 FETCH (FLAGS (\Deleted \Seen))
d OK Store completed (0.001 + 0.000 secs).

Now that we have stored the Deleted flag on message 2, let’s expunge the Inbox:

e expunge
* 2 EXPUNGE
e OK Expunge completed.

Which, as you can see, has resulted in message 2 being deleted forever.

At this point, I’ve shown you how to login to an IMAP server, list a set of folders in your personal namespace, select the Inbox, perform some basic searching, retrieve a message or message parts, mark a message for deletion and finally delete.

I encourage you to read over RFC 3501 to get a feel for all of the amazing IMAP commands on offer. In a follow up post we will explore some more useful commands.

Find This Useful?

We’re taking ideas for future email-related posts. Suggestions invited here.

New to atmail?

If email is not your core business and you’d like someone else to worry about IMAP, SMTP and POP, we can help.

With 20 years of global, white label, email expertise serving telecommunications and hosting providers across every continent, you can trust us to deliver white label, email solutions that are stable, secure and scalable.

We power 170 million mailboxes and offer user-friendly, cloud hosted email with 99.99% uptime and your choice of US or (GDPR compliant) EU data centres.

Or, if you want to stay in-house, we offer on-premises webmail and/or mail server options.

Talk to us today.

Share This Post

Products

Support

Resources

Compliance

Company

IMAP 101: Manual IMAP Sessions

What is IMAP?

Connecting to an IMAP Server

A Quick Note on Tags

1. Plaintext or Telnet (or Netcat)

2. SSL

Authenticating to IMAP

Authenticating via Login Method

Authenticating via SASL LOGIN Method

Authenticating via SASL PLAIN Method

Special Credit: Authenticating via SASL PLAIN as an Admin User

Retrieving Emails and Modifying the Inbox

Namespace

Select the Inbox

Fetch Messages

Delete a Message

Find This Useful?

New to atmail?

IMAP 101: Manual IMAP Sessions

What is IMAP?

Connecting to an IMAP Server

A Quick Note on Tags

1. Plaintext or Telnet (or Netcat)

2. SSL

Authenticating to IMAP

Authenticating via Login Method

Authenticating via SASL LOGIN Method

Authenticating via SASL PLAIN Method

Special Credit: Authenticating via SASL PLAIN as an Admin User

Retrieving Emails and Modifying the Inbox

Namespace

Select the Inbox

Fetch Messages

Delete a Message

Find This Useful?

New to atmail?

More Posts

Further Contemplations on Email’s Future with a Nod to its History

ChatGPT and the History of Email