Earlier today we have made some significant changes to the tar-pitting mechanism under ExchangeDefender. The new mechanism is designed to reject messages from hosts that do not follow the proper RFC SMTP dialog and attempt to smash tar-pitting. More on the basic concept of tar-pitting is described here.

The Problem

While tar-pitting is great for throttling remote mail servers and reducing their ability to efficiently deliver a lot of messages, the concept only applies against botnet servers that are attempting to deliver mail in bulk. Anotherwords, tar-pitting is only effective against servers that are concerned about getting the message out as fast as possible. By delaying the SMTP greeting banner, in theory, the remote mail server would have to wait a pre-determined amount of time before starting to send mail. Many open connections at once would overload a single node.

However, spammers no longer exclusively use single nodes in a full force attack. They use the botnet concept by load balancing their broadcasts through multiple servers. As such, those servers connect every few minutes and only relay a single message. By doing so its hard to blacklist them immediately because their overall reputation does not have enough data to be determined. These botnets are designed to bypass tar-pitting by opening a connection and sending data as soon as the connection is opened.

The conversation looks somewhat like this:

Trying xx.xx.xx.xx.exchangedefender.com.

Escape character is ‘^]’.
ehlo spamming-idiot.org
mail from: spammer@spammer.org
rcpt to: vlad@ownwebnow.com
data
Subject: Get a college diploma.
Ohio State University may be a loser but they’ll give you a Ph.D in nuclear physics based on your life experience.
.

Now the (target) tar-pitting mail server has accepted the connection but it has never sent the SMTP greeting. However, it will process the message as soon as its tar-pitting interval passes, thus in part bypassing the tar-pitting and delivering the message. Not good.

Notice that the client above did not wait for the 200 greeting banner, did not wait for the 250 Hello, did not wait for the server to acknowledge the recipient and the sender. They just wrote to the socket and waited. Now even though this does diminish the spammers performance a little (by taking 5 seconds to deliver the message) the message still gets delivered. That’s a problem.

The Solution

The solution is fairly simple: Drop connections with mail servers that are not adhering to RFC. The second the mail server issues a command before the 2.2.0 hostname greeting banner it will get dropped, logged and its tar-pitting interval extended.

Instead of a tar-pitting process that delays the connection a few seconds, this process allows for a connection immediately but delays the SMTP greeting banner a few seconds. As such, it can eliminate server load caused by spammers that think they have found a way around tar-pitting.

We ran this in testing on our production systems and have found 0 false positives over the course of one week. All hosts that were rejected were also on multiple RBLs. The implementation is transparent to the user and administrator and introduces a random (less than 5) second pause on all connections that do not have a reputation rating with ExchangeDefender. Less spam, less stuff to review, less bandwidth and less stress for you.