Dear Mr Laguna, IMAP is Not the Right Protocol for Chat

I was surprised and disheartened last month when I read an article entitled, Meet the open sorcerers who have vowed to make Facebook history: Checking in on how IMAP could help folk throw off Zuck’s yoke, in which Rafael Laguna (CEO of Open-Xchange) announced a plan to extend IMAP for use as group chat.

I have a lot of respect for the extent to which Mr Laguna has supported open source software and provided solutions to assist the businesses which adopt them. But I feel his IMAP mission is misdirected, because when a protocol is extended beyond its core domain, it risks making the implementation of the protocol both difficult and unreliable. Especially when, as even he says in the article, “So much time is spent replicating old models that have no chance of success whatsoever”. 

 

How IMAP was Good for Email

IMAP is the protocol that defines communication between mail servers and the email clients. Yet today’s email clients and devices suffer in functionality, despite being used by nearly everyone who’s connected to the internet.

When using IMAP, users have to wait a while for an email sent on their computer to appear as ‘sent’ item on their mobile device. In addition, their mobile device suffers the power drain of having to maintain an open connection, or by polling at regular intervals.

Apple has implemented an ugly work around, in an attempt to prolong battery life, by utilising a push notification service on their iPhone and iPad products. But the complexities required to implement IMAP in both user and server side software has resulted in user difficulties and a lack of software options, along with a rigid user experience.

So, when atmail decided to improve the quality of our email service, we chose to develop an implementation of a proposed, new, open source protocol – rather than try to extend beyond the capacity of IMAP. The new JMAP protocol solves the existing issues and is modular enough to take advantage of future technology. JMAP makes email better.

(And yes, rather than keep the results of our great work in-house, we will share the benefits with the world. This meets our mission of improving the email experience, as well as acting as a catalyst for standardisation and adoption. atmail have supported open protocols from the very beginning and we will in the future be open sourcing the code we have developed for the JMAP protocol that powers our atmail suite and mobile products, to further strengthen the JMAP community.)

 

Keeping Protocols Focussed on their Job

IMAP is a complex protocol that was developed during a period where the operating paradigm was a single desktop client (or at most, two operating clients) which would maintain a single, long (and many short-lived) connections. But today, the number of client applications connected simultaneously has exploded via desktops, laptops, phones, tablets and even wearables.

On top of all this, the usage of email storage has changed, as people have stopped using mail servers like a mailbox that they empty into their only computer. They have instead started using mailboxes as an online database (and I dare say, junk yard) that all of their devices connect to and keep the data in sync.

But writing a new standard, getting industry support and having it accepted is a difficult task – so, hats off to FastMail for taking this giant step forward for the industry and proposing the JMAP protocol – and keeping it focussed. This is something that has been guiding the development of JMAP from the beginning. One of the hardest things to achieve in a protocol is meeting all the existing needs whilst being future-ready and not allowing scope creep. The only way to achieve this is to have a very solid understanding of what the protocol is meant to achieve and what is not an appropriate use for a protocol.

So, when I saw Mr Laguna announce a plan to extend IMAP for use as group chat, I was surprised and disheartened. The atmail team has worked hard to solve the high latency of server-to-client email communication in a modern web-enabled world, by using more appropriate web-based technologies. IMAP has neither the speed, nor the appropriate multi-device syncing capabilities that must be the foundation of a successful group chat service, particularly in the modern world of mobile devices. IMAP is certainly a weird protocol choice by Mr Laguna, especially given the existence of an appropriate and established protocol,  the Extensible Messaging and Presence Protocol (XMPP). XMPP, originally designed by the Jabber community in 1999, is perfect for instant messaging because it focuses on near real-time communication, as well as providing information on user presence and allowing global interoperability – just like email!

 

Then Why Turn to IMAP for Chat?

Mr Laguna is the CEO of Open-Xchange, whose web-based email software includes Dovecot (owned by Open-Xchange), an open-source IMAP and POP3 server for Linux/UNIX-like systems. With a global market share of around 68-75% of all IMAP servers, it makes sense to leverage this advantage by expanding into the popular instant messaging market. It is just a pity that IMAP is not a good fit and that an industry leader like Open-Xchange would even consider this move.

Like myself, Mr Laguna’s focus is broader than turning an income. He’s passionate about open-source and believes that users should own their data and be independent of the service providers that manage it. Mr Laguna has stated, “I’ve got 30 years of email, and I’ve changed provider dozens of times in the meantime. A service cannot be trusted if it’s not available from many providers.” But this is not the case with social networks or proprietary messaging systems. You can’t protest Facebook’s management of your privacy by taking all of your messages and posts with you over to Google+ or some other service provider – it’s just not possible.

 

A Protocol for Social Networking

It is Mr Laguna’s plan to change that, by developing an open protocol for instant messaging and social networks. With his current influence over global IMAP servers, I feel that the IMAP protocol is being considered for even further misdirected extension and overload.

As rhodey, a developer behind Flock, stated on GitHub in regard to overloading WebDav, “Choosing the wrong protocol led to unreasonable server costs, at this time Flock is nowhere near affordable.”

And Markus Unterwaditzer, creator of vdirsyncer, supports this view with a similar experience, “The fact that CalDAV and CardDAV are based on WebDAV has fatal downsides in practice. I guess the idea was that you could just use an existing WebDAV client library to access your calendar data in an easy way. In practice, all of this complexity makes servers really hard to implement properly.”   

Where a protocol is already being overloaded to cope with the modern world, to the extent where most implementations include proprietary workarounds, migrating to a new protocol to achieve those tasks is a more stable and more interoperable option.

In situations like IMAP functioning so poorly that the licensed proprietary solution ActiveSync dominates the market, it is clearly time to support the adoption of a new open protocol that addresses these needs in the email space. And, let’s not all forget, email at it’s heart is an asynchronous communication medium and was never meant to synchronous – despite how many people believe it should be and how the industry continues to try and support this dream.

 

A Better Protocol Provides a Better Service

Again, I have a lot of respect for the extent to which Mr Laguna has supported open source software and provided solutions to assist the businesses which adopt them.

So, it is my hope that Mr Laguna will recognise the value in supporting the adoption of a new protocol like JMAP, to reduce the complications that arise from implementing IMAP for client synchronisation. And by providing a better, faster and richer email experience for his clients, move forward in his mission to provide social networking alternatives, where the user retains ownership and portability of their data.

Mr Laguna, please just support XMPP for your chat aspirations and abandon this notion of extending IMAP – please, for the sake of the messaging world.