Greylisting with MIMEDefang

I’ve written a simple greylisting facility within MIMEDefang. It uses a database — PostgreSQL in my case, but anything supported by Perl’s DBI module will work.

This article assumes that you know what greylisting is and why you want to implement it, that you have a working recent version of MIMEDefang, and that you are moderately comfortable in editing its configuration file mimedefang-filter.

The implementation is straight from the greylisting white paper and uses pretty basic Perl coding as I’m a pretty basic Perl programmer. The only embellishment to the code is to strip off the last octet of the sending relay’s IP address. This is to reduce the likelihood of repeated delays with large ISPs where a mail may originate from a different IP each time. The less severe condition that I assume here is that it should originate from within the same /24 each time.

Preparation

First, you need to set up a database table. Here, I am using a bespoke database called mail as I have several mail-related tables in it. Create a table called greylist with eight columns:

  1. ip: a character varying column 11 characters wide
  2. sender: a character varying column 255 characters wide
  3. recipient: a character varying column 255 characters wide
  4. passed: an integer column
  5. blocked: an integer column
  6. created: an integer column
  7. blockuntil: an integer column
  8. expires: an integer column

For optimum performance, create a multi-column index on ip, sender and recipient. If you can create multi-column unique primary keys, do that on the same trio of columns.

The code

Initialization

First, we need to open a connection to our database. At the very top of the file, we need to tell MIMEDefang to use the DBI module. Ideally, we should avoid opening and closing connections as much as possible, so we use MIMEDefang’s filter_initialize subroutine to get a handle that will persist for the lifetime of the Perl slave. We also prepare the database queries that will be used in the greylisting subroutine.

If you already have a filter_initialize function, add the code below to it; otherwise, create one from scratch. It doesn’t matter where you put it within mimedefang-filter but it makes sense to have it near the top.

See the documentation for your database’s DBD module to find out the correct connect incantation for your setup.

use DBI;

sub filter_initialize {
  our $dbh = DBI->connect("dbi:Pg:dbname=your_db_name","your_user_name");
  our $glc = $dbh->prepare ("INSERT INTO greylist VALUES (?,?,?,0,1,?,?,?);");
  our $glr = $dbh->prepare ("SELECT blockuntil FROM greylist WHERE ip = ? AND sender = ? AND recipient = ?;");
  our $glu = $dbh->prepare ("UPDATE greylist SET expires = ? WHERE ip = ? AND sender = ? AND recipient = ?;");
  our $glbi = $dbh->prepare ("UPDATE greylist SET blocked = blocked + 1 WHERE ip = ? AND sender = ? AND recipient = ?;");
  our $glpi = $dbh->prepare ("UPDATE greylist SET passed = passed + 1 WHERE ip = ? AND sender = ? AND recipient = ?;");
}

Checking the greylist

I run the greylist check in filter_recipient, which is the earliest point in the transaction where we have all three pieces of data we need.

Obviously, if you have a filter_recipient subroutine already, simply add the greylist call to it after any code that filters based on other conditions: there is no point greylisting stuff you are only going to reject later!

No provision for whitelisting is made here: you can do so yourself in filter_recipient by returning a CONTINUE array ahead of the greylist call.

If you do not have a filter_recipient subroutine, make sure you use the -t flag to mimedefang-multiplexor in your start-up script, or the subroutine will not get called.

filter_recipient is required to return a two-element array containing one of CONTINUE, TEMPFAIL and REJECT in the first element, and a message in the second. Our greylist subroutine will return such an array itself, so we can simply pass on its return value. Clear?

sub filter_recipient {
  my ($recipient, $sender, $ip, $hostname, $first, $helo, $rcpt_mailer, $rcpt_host, $rcpt_addr) = @_;
  return greylist($ip, $sender, $recipient);
}

The central subroutine

This is the subroutine that does all the work. First, it trims the IP address down to the first three octets only, and converts sender and recipient to lowercase. Then, it runs the $glr prepared query to see if the ip/sender/recipient triple is already in the database.

If it is, the query returns the timestamp representing the blockuntil time for that record. If that is still in the future, the blocked count is incremented and the subroutine returns a temporary failure. If the blockuntil time has passed, the expires time is updated, the passed count is incremented, and the subroutine returns success.

If the record is not found in the database, it is added and a temporary failure is returned. It is theoretically possible for a race condition to occur if two processes nearly simultaneously check for a non-existent triple, and then both try to create one. If your database setup has defined a triple as a unique key, the second write attempt will fail. If the database doesn’t restrict it, you’ll end up with a duplicate key, which is harmless.

I use times of 120 seconds blocking, and 36 days (3,110,400 seconds) record expiry. These values can be changed within the code if required. See the greylisting white paper for rationale for the values used here.

sub greylist {
  my ($ip, $sender, $recipient) = @_;
  my ($ip3) = ($ip =~ /^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})/);

  $sender = lc($sender);
  $recipient = lc($recipient);

  $glr->execute($ip3, $sender, $recipient);

  if ($glr->rows) {
    my ($blockuntil) = $glr->fetchrow_array();
    if ($blockuntil > time) {
      $glbi->execute($ip3, $sender, $recipient);
      return ('TEMPFAIL', "Please try later: greylisted for " . ($blockuntil - time) . " seconds.");
    }
    $glu->execute(time + 3_110_400, $ip3, $sender, $recipient);
    $glpi->execute($ip3, $sender, $recipient);
    return ('CONTINUE', "ok");
  }
  else {
    $glc->execute($ip3, $sender, $recipient, time, time + 120, time + 3_110_400);
    return ('TEMPFAIL', "Please try later: greylisted for 120 seconds.");
  }
}

Cleaning up

To clean up the resources we took in filter_initialize, we need some code in filter_cleanup (again, you may need to create this subroutine; again, it does not matter where it goes). Your database will not burst into flames if you omit this step, but let’s do things properly here, OK?

sub filter_cleanup {
  $glc->finish;
  $glr->finish;
  $glu->finish;
  $glbi->finish;
  $glpi->finish;
  $dbh->disconnect;
}

Purging the database

The astute reader may have noticed that old records are never cleared out. It would be possible to include a deletion query (completing the CRUD), but this would consume unnecessary resources if we ran it every time greylist was called. Instead, a simple Perl script called daily through crontab does the job. I put mine in /etc/mail/purge-greylist and made it executable, and added a line to my crontab to run it early every morning.

#!/usr/bin/perl

use strict;
use DBI;

my $dbh = DBI->connect("dbi:Pg:dbname=your_db_name","your_user_name");
my $gld = $dbh->prepare("DELETE FROM greylist WHERE expires < " . time . ";");

$gld->execute;
$gld->finish;
$dbh->disconnect;

1;