How to use the new logging facility?

Last week, logging facility has been added to VMime messaging services. This will allow the developer to trace what is sent and received on the connection between client and server, and may help debugging. We will see in this post how to use this feature, which is supported for IMAP, POP3 and SMTP protocols.

First, you have to create your own tracer, which must implement the vmime::net::tracer interface:

class myTracer : public vmime::net::tracer
{
public:

    myTracer(const vmime::string& proto, const int connectionId)
        : m_proto(proto), m_connectionId(connectionId)
    {
    }

    // Called by VMime to trace what is sent on the socket
    void traceSend(const vmime::string& line)
    {
        std::cout << "[" << m_proto << ":" << m_connectionId
                  << "] C: " << line << std::endl;
    }

    // Called by VMime to trace what is received from the socket
    void traceReceive(const vmime::string& line)
    {
        std::cout << "[" < < m_proto << ":" << m_connectionId
                  << "] S: " << line << std::endl;
    }

private:

    const vmime::string m_proto;
    const int m_connectionId;
};

Also create a factory class, used to instanciate your tracer objects:

class myTracerFactory : public vmime::net::tracerFactory
{
public:

    vmime::shared_ptr <vmime::net::tracer> create
        (vmime::shared_ptr <vmime::net::service> serv,
         const int connectionId)
    {
        return vmime::make_shared <myTracer>
               (serv->getProtocolName(), connectionId);
    }
};

Next, we have to tell VMime to use it. When you create your service (either store or transport), simply call the setTracerFactory on the service and pass an instance of your factory class:

 vmime::shared_ptr <vmime::net::transport> store =
     session->getStore("imaps://user:password@imap.myserver.com");

 // Enable tracing communication between client and server
 store->setTracerFactory(vmime::make_shared <myTracerFactory>());

That's all! Now, everything which is sent on/received from the socket will be logged using your tracer object. Here is an example of a trace session for IMAP:

[imaps:1] S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready.
[imaps:1] C: a001 AUTHENTICATE PLAIN
[imaps:1] S: + 
[imaps:1] C: {...SASL exchange: 52 bytes of data...}
[imaps:1] S: a001 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SPECIAL-USE QUOTA] Logged in
[imaps:1] C: a002 LIST "" ""
[imaps:1] S: * LIST (\Noselect) "." ""
[imaps:1] S: a002 OK List completed.
[imaps:1] C: a003 CAPABILITY
[imaps:1] S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SPECIAL-USE QUOTA
[imaps:1] S: a003 OK Capability completed.
[imaps:1] C: a003 SELECT INBOX (CONDSTORE)
[imaps:1] S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk NonJunk JunkRecorded $MDNSent NotJunk $Forwarded Junk $Junk Forwarded $MailFlagBit1)
[imaps:1] S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk NonJunk JunkRecorded $MDNSent NotJunk $Forwarded Junk $Junk Forwarded $MailFlagBit1 \*)] Flags permitted.
[imaps:1] S: * 104 EXISTS
[imaps:1] S: * 0 RECENT
[imaps:1] S: * OK [UNSEEN 6] First unseen.
[imaps:1] S: * OK [UIDVALIDITY 1268127585] UIDs valid
[imaps:1] S: * OK [UIDNEXT 32716] Predicted next UID
[imaps:1] S: * OK [HIGHESTMODSEQ 148020] Highest
[imaps:1] S: a003 OK [READ-WRITE] Select completed.

Please note that no sensitive data (ie. login or password) will be traced. Same, "blob" data such as message content or SASL exchanges will be logged as a marker which indicates how many bytes were sent/received (eg. "{...SASL exchange: 52 bytes of data...}").

New feature: simultaneously get and fetch messages

We have released a new feature for IMAP stores: get and fetch messages in a single operation. This will avoid one round-trip to the server.

Let’s say we want to get the flags for the messages UID 2 to 6. Until today, we used to write the following code:

vmime::shared_ptr <vmime::net::folder> f = /* ... */;

vmime::net::messageSet mset(vmime::net::messageSet::byUID(2, 6));

vmime::net::fetchAttributes attribs;
attribs.add(vmime::net::fetchAttributes::FLAGS);

std::vector <vmime::shared_ptr <vmime::net::message> >
    msgs = f->getMessages(mset);

f->fetchMessages(msgs, attribs);

// ...do something with msg[n]->getFlags()...

This resulted into the following IMAP client-server dialog, with two FETCH commands in sequence:

C: a005 UID FETCH 2:6 UID                <-- getMessages()
S: * 1 FETCH (UID 2 MODSEQ (1268172))
S: * 2 FETCH (UID 3 MODSEQ (1268417))
S: * 3 FETCH (UID 4 MODSEQ (1268417))
S: * 4 FETCH (UID 5 MODSEQ (1268312))
S: * 5 FETCH (UID 6 MODSEQ (1268417))
S: a005 OK Success
C: a006 FETCH 1:5 (FLAGS)                <-- fetchMessages()
S: * 1 FETCH (MODSEQ (1268172) FLAGS (NotJunk $NotJunk \Seen))
S: * 2 FETCH (MODSEQ (1268417) FLAGS (NotJunk $NotJunk \Seen))
S: * 3 FETCH (MODSEQ (1268417) FLAGS (NotJunk $NotJunk \Seen))
S: * 4 FETCH (MODSEQ (1268312) FLAGS (NotJunk $NotJunk \Seen))
S: * 5 FETCH (MODSEQ (1268417) FLAGS (NotJunk $NotJunk \Seen))
S: a006 OK Success

With the new getAndFetchMessages() method, we can now write:

vmime::shared_ptr <vmime::net::folder> f = /* ... */;

vmime::net::messageSet mset(vmime::net::messageSet::byUID(2, 6));

vmime::net::fetchAttributes attribs;
attribs.add(vmime::net::fetchAttributes::FLAGS);

std::vector <vmime::shared_ptr <vmime::net::message> >
    msgs = f->getAndFetchMessages(mset, attribs);

// ...do something with msg[n]->getFlags()...

Here is now the dialog between the client and the server:

C: a006 UID FETCH 2:6 (FLAGS UID MODSEQ) 
S: * 1 FETCH (UID 2 MODSEQ (1268172) FLAGS (NotJunk $NotJunk \Seen))
S: * 2 FETCH (UID 3 MODSEQ (1268417) FLAGS (NotJunk $NotJunk \Seen))
S: * 3 FETCH (UID 4 MODSEQ (1268417) FLAGS (NotJunk $NotJunk \Seen))
S: * 4 FETCH (UID 5 MODSEQ (1268312) FLAGS (NotJunk $NotJunk \Seen))
S: * 5 FETCH (UID 6 MODSEQ (1268417) FLAGS (NotJunk $NotJunk \Seen))
S: a006 OK Success

That is, one round-trip to the server saved. Please note that this will work with any fetch attribute, and should also work with any IMAP-compliant server!

New supported SMTP extensions

Hello VMime users,

In the last weeks, we added support for the following SMTP extensions :

  • RFC-6531: SMTP Extension for Internationalized Email (this is the continuation for the work that has been done for Internationalized Email Support)
  • RFC-3030: CHUNKING extension, for transmission of large messages
  • RFC-3030: SMTP Service Extension for Message Size Declaration

For now, few SMTP servers claim to support these features, but this is changing and we have to be ready for that. We are working on adding more features to VMime, keep on touch with us!

Internationalized Email Support

Hi there!

Today, we committed a big change into VMime codebase. We implemented preliminary support for RFC-6532 (Internationalized Email Headers), which is the (future) replacement for RFC-2047. We added support for IDN (Internationalized Domain Name) and EAI (Email Address Internationalization).

As a lot of MUAs and MSAs do not support this yet, the feature is not enabled by default (and, to be honest, we need to test this thoroughly in VMime before using it in production code). To enable it from your code, just configure the default generation context:

vmime::generationContext::getDefaultContext()
    .setInternationalizedEmailSupport(true);

In the meantime, we also added message generation/parsing context and charset conversion options, to provide a better control to the user on how the messages are parsed/generated. Have a look at Doxygen-generated documentation to know more about this.

CMake and OpenSSL support!

Hi, here are some news from the front!

We migrated the build system to CMake. SCons is still usable, but should be used only for development purposes. Please note that autotools scripts should now be generated with CMake, as it has been removed from SCons script.

Thanks to the help of Mehmet Bozkurt, VMime now supports OpenSSL for SSL/TLS features, in addition to the existing support for GnuTLS.

We hope these new features will make deployment of VMime easier.

Version 0.9.1 released

Hi all!

We are proud to announce VMime 0.9.1 release, which includes all bug fixes and features from the last two years (yeah, I know this looks like eternity, but don’t worry, VMime is still regularly maintained!).

  • Fixed a lot of compilation issues on some platforms
  • Fixed message parsing and generation issues
  • Better support for encoding
  • Dual-licensing

Thanks to all who contributed to this release, and those using VMime! We are working hard to provide the Open Source world with a powerful and reliable mail library since more than 10 years! ;-)

Connection time out

By popular demand, it is now possible to handle time out during connection to IMAP, POP3 and SMTP servers.

There is nothing to change in your code to enjoy this new feature, at least if you already set a time out handler on the store (vmime::net::store::setTimeoutHandler). The same time out handler is used for connection and for data exchange after the connection has been opened.

Please note: for now, this is only available in VMime SVN, revision 559, and for POSIX platforms (this feature is not yet supported on Windows).

New feature: get parsed message from store

Hi VMime users!

Let’s focus today on a new feature in VMime 0.9. Until now, when you wanted to “explore” the contents of a vmime::net::message (that is, a message hosted on a IMAP or a POP3 store), you had to extract it entirely, or to use low-level fetch() and extract() functions on the message object.

Since the current development release of May 18th, there is a new helper function that will let you access the remote message like a “normal” vmime::message, and actually download the message data only when it is required (this only works for IMAP).

// Connect to the IMAP store
vmime::ref <vmime::net::session> sess = vmime::create <vmime::net::session>();

vmime::utility::url storeURL("imap://username:password@imap.example.com");

vmime::ref <vmime::net::store> store = sess->getStore(storeURL);
store->connect();

// Open the INBOX
vmime::ref <vmime::net::folder> folder = store->getDefaultFolder();
folder->open(vmime::net::folder::MODE_READ_WRITE);

// Get the first message in the INBOX
vmime::ref <vmime::net::message> msg = f->getMessage(1);

// Construct the parsed message (a few data, header and structure,
// is actually downloaded from the IMAP server)
vmime::ref <vmime::message> parsedMsg = msg->getParsedMessage();

// Here, extract() will actually download message data from the server
vmime::utility::outputStreamAdapter out(std::cout);
parsedMsg->getBody()->extract(out);

That’s all folks! Update your source tree to the latest SVN head to enjoy this new feature.

Connecting to GMail SMTP/IMAP

Some users fequently ask me how to connect to GMail SMTP service with VMime. You have to connect to server using SMTP protocol (not SMTPS), and set the connection.tls property to true to initiate a secured connection using STARTTLS.

The following code is known to work:

vmime::utility::url url("smtp://smtp.gmail.com");
vmime::ref <vmime::net::transport> tr = session->getTransport(url);
tr->setProperty("connection.tls", true);
tr->setProperty("auth.username", "gmail-login");
tr->setProperty("auth.password", "gmail-password");
tr->setProperty("options.need-authentication", true);
tr->setCertificateVerifier(yourCertificateVerifier);

To connect to IMAP on GMail, use the following code:

vmime::utility::url url("imaps://login:password@imap.gmail.com:993");
vmime::ref <vmime::net::store> store = session->getStore(url);
store->setCertificateVerifier(yourCertificateVerifier);
store->connect();