A typical organization runs an email system to accept mail addressed to local users and to send local users' mail to each other and to the outside world.
I like to use the Postfix email system, because it is relatively easy to configure and has a good record for security. More information about Postfix is available from the Postfix website.
This whitepaper discusses how to set up the Postfix email system to handle your organization's email needs. Special attention is focused running Postfix on Debian Linux, getting Postfix to co-operate with a Cyrus IMAP server, and configuring authentication-based relay authorization.
Mail transport agents (MTAs) like Postfix speak to each other using SMTP (Simple Mail Transport Protocol). To understand how MTAs work, and sometimes don't work, it will help to have a very basic understanding of this protocol. If you telnet to the smtp port (port 25) of a mail server, you can speak with an MTA directly in its native language. Here is an example of how to send [email protected] an email from [email protected].
$ telnet smtp.example.com smtp
Trying 192.0.34.72...
Connected to smtp.example.com.
Escape character is '^]'.
220 smtp.example.com ESMTP Postfix (Debian/GNU)
HELO smtp.somewhere.net
250 smtp.example.com
MAIL From: [email protected]
250 Ok
RCPT To: [email protected]
250 Ok
DATA
354 End data with.
Hello, Bob!
Love, Alice.
.
250 Ok: queued as F169C23068
QUIT
221 Bye
Connection closed by foreign host.
Each MTA that handles a mail adds headers containing information about the SMTP session. For example, when it is stored at its destination, the full header and body of the mail we sent above looks like this.
Received: from smtp.somewhere.net (smtp.badguy.net [])
by smtp.example.com (Postfix) with SMTP id F169C23068
for; Tue, 6 Aug 2002 12:02:15 -0700 (PDT)
Message-Id <[email protected]>
Date: Tue, 6 Aug 2002 12:02:15 -0700 (PDT)
From: [email protected]
To: undisclosed-recipients:;
Hello, Bob!
Love, Alice.
The postfix source is available for download from the postfix website, but Debian provides well-maintained packages which will satisfy our needs.
# aptitude install postfix
# aptitude install postfix-ldap postfix-tls
Postfix configuration files live in the directory /etc/postfix/. The main configuration file for postfix is /etc/postfix/main.cf, which is organized into simple key = value directives, one per line. Since Postifx is very good at guessing reasonable defaults, it is actually possible to have a working mail server with a completely empty main.cf, but there are probably at least a couple of values you will want to set explicitly.
myhostname = smtp.example.com
mydomain = example.com
The first line sets $myhostname to a FQDN (fully qualified domain name) for your mail server. This is the name that postfix will report to fellow SMTP daemons when exchanging mail. Other SMTP daemons may take umbrage if your SMTP daemon reports a false or fake FQDN, so make sure $myhostname really does resolve to your server's IP address. By default, $myhostname is determined via the hostname() API, and $mydomain is set to the domain part of $myhostname.
If you want to receive incoming mail, you will need to tell Postfix the domain names (the part of an email address after the @) it should interpret as local users.
mydestination = $mydomain $myhostname localhost.$mydomain
The $mydestination parameter lists the domains for which the SMTP server will treat mail as destined for a local user. In our example, we have told postfix that all mail to addresses of the form [email protected] and [email protected] is to be accepted for local delivery.
When postfix recieves a mail to an address not contained in the mydestination parameter, it assumes that it is being asked to relay the mail to the appropriate destination. If postfix's configuration allows it to relay the mail, it will attempt to contact the computer named in the mail's address.
Most of the computers within your organization should not accept mail for local delivery, but instead pass everything on to a mail hub. It is easy to configure Postfix to do this.
mydestination =
relayhost = smtp.example.com
Next, you need to tell Postfix where to put the mail it accepts for local delivery. By default, it will put mail for username into the file /var/spool/mail/username. This is the traditional storage location for mail on Unix systems, and most console-based Unix email clients get their mail from there.
The trouble with this approach is, pretty much only console-based Unix email clients get their mail from there. Very few users today, even Unix users, use such clients to read their email. Most users like to access their mail across a network using GUI-based clients which speak the IMAP or POP protocols. To work with such clients, you need to configure Postfix to hand mail off to an IMAP or POP server.
We use Cyrus-IMAP for local mail storage. It is well-maintained and open-source, supports POP and well as IMAP access, and has several useful additional features such as native SSL support and support for sieve filtering.
Cyrus, as well as a number of other modern mail storage systems, use LMTP (local mail transport protocol) to accept incoming mail. Fortunately, Postfix speaks LTMP, too.
If Cyrus is on the same computer as Postfix, you can use a filesystem socket for local delivery. For example,
mailbox_transport = lmtp:unix:/var/run/cyrus/socket/lmtp
# adduser postfix mail
If Cyrus is on a different computer, use an LMTP network socket.
mailbox_transport = lmtp:inet:imap.example.com
Aliases are alternative names for local users. If Postfix is sent a mail for [email protected], it will check its alias tables to see whether an-alias maps to one or more existing local users before it returns a bounce notice.
Certain aliases are required by tradition and RFC to exist for any organization. These are:
alias | role |
---|---|
sysadmin | the system administrator |
postmaster | the email administrator |
webmaster | the website administrator |
abuse | address to report hackers, spammers |
We will show you two ways to implement alias tables: one which defines aliases in the local file /etc/aliases, and other which stores them in an LDAP database. To let Postfix know about these two alias tables, we use the following directive.
alias_maps = hash:/etc/aliases, ldap:ldapdata
Our /etc/aliases file contains the basic, required aliases.
root: user1, user2
sysadmin: root
postmaster: root
abuse: root
webmaster: root
www: root
Postfix actually uses a hash of the aliases file, instead of consulting it directly, in order to speed lookup. To generate the hash, enter
# postalias /etc/aliases
ldapdata_server_host = ldap.example.org
ldapdata_search_base = dc=example,dc=org
ldapdata_query_filter = (mailLocalAddress=%s)
ldapdata_result_attribute = mailRoutingAddress
When handed a mail, Postfix compares the domain name in the RCPT "To:" data (the part after the @) with $mydestination to determine whether the mail should be delivered to a local user or relayed to a remote host.
In the early days of email, SMTP servers would happily relay any mail handed to them; it was the neighborly thing to do. An SMTP server willing to relay any mail handed to it is called an "open relay". These days, the internet is a less neighborly place, full of bandwidth thieves and spammers who are only too happy to use your open relay for their own nefarious purposes. Running an open relay is not only unneighborly; it is likely to get your organization blacklisted as a source of spam. This section covers how to configure Postfix to ensure that only authorized mail is relayed through your server.
Relaying is controlled by the smtpd_recipient_restrictions directive, which accepts as its value a stack of restrictions which are evaluated, in order, until one returns either REJECT or PERMIT. (If you have experience with PAM stacks or with chains of firewall rules, this concept will be familiar.)
The simplest form of relay control is accept mail for relay only from IP addresses owned by your organization, or otherwise known to be trustworthy.
mynetworks = 127.0.0.0/8 192.0.34.0/8
relay_domains =
smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, permit
IP-based relay control is usually effective (the exception being a rogue user of a trusted machine), but often unnecessarily restrictive. Users with home internet access will likely want to use your smtp server to send email from home, which requires you to add their home IP address (or even large IP address blocks, if their provider uses DHCP) to $mynetworks. Users who travel will likely want to send mail from wherever they go, which is completely impossible using an IP-based scheme.
An effective alternative to IP-based relay control is authentication-based relay control. Before relaying a mail, Postfix requests a username and password, then sends the mail only if the username and password are valid. Authenticated users are allowed to send mail from any computer anywhere. Most modern mail clients support SMTP authentication.
Setting up authentication-based relay control is significantly more complex than setting up IP-based relay control, not least because you will want to ensure that communications are encrypted before you allow a password to be transmitted. Still, until you set up authentication-based relay, you will be subject to ceaseless pressure from users to make your relay ever more open, until some day you bite the bullet and decide to read this section.
The first step is to insert permit_sasl_authenticated into the smtpd_recipient_restrictions list.
smtpd_recipient_restrictions = permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination, permit
Before configuring Postfix to support authentication, we need to configure it to support public key encryption To do so, create a certificate and key file for your mail server as described in our Digital Certificates whitepaper. Then tell Postfix where to find these files.
smtpd_use_tls = yes
smtpd_tls_cert_file = /etc/postfix/smtp.example.crt
smtpd_tls_key_file = /etc/postfix/smtp.example.key
Now we can enable authentication, with the requirement that the TLS encryption layer be active before authentication occurs.
smtpd_sasl_auth_enable = yes
smtpd_tls_auth_only = yes
Postfix now knows that it is should support the SMTP AUTH command, but it does not yet know how to check the usernames and passwords clients give it for validity. To get authentication working, you need to specify a method for the SASL library to check usernames and passwords. The method most likely to be familiar to Linux users is PAM, which works just fine. For SASL version one, it was possible to specify PAM to Postfix directly as the SASL authentication mechanism. The current version of SASL does not support this; instead, we must tell Postfix to use saslauthd, the SASL authentication daemon, to check passwords, and then tell saslauthd to use PAM in turn.
To obtain saslauthd, install the sasl2-bin package
# aptitude install sasl2-bin
Next, tell Postfix to use saslauthd for password checking.
pwcheck_method: saslauthd
mech_list: plain login
# adduser postfix sasl
# /etc/init.d/saslauthd restart
Finally, tell saslauthd to use PAM for authentication.
START=yes
MECHANISMS="pam"
auth required pam_ldap.so
account required pam_ldap.so
session required pam_ldap.so
Unsolicited bulk email, colloquially called spam, is an increasingly problematic nuisance for both everyday email users and system administrators. Postfix offers a number of configuration directives you can use to try to catch and reject incoming spam.
The heart of Postfix's spam control system is a gauntlet of four restriction lists. The restriction lists reject or permit mail based on configurable criteria. The lists are named smtpd_*_restrictions, where the * is one of:
client | restricts which clients may connect to smtpd |
helo | restrictions based on HELO (or EHLO) greeting |
sender | restrictions based on MAIL From: data |
recipient | restrictions based on RCPT To: data |
Some restrictions may appear on any list.
permit | PERMIT all mail |
reject | REJECT all mail |
The smtpd_client_restrictions list makes decisions based on the IP address of the connecting client. Possible list entries include:
permit_mynetworks | PERMIT if IP is in $mynetworks |
reject_unknown_client | REJECT if IP has no PTR in DNS |
reject_rbl_client | REJECT if IP is blacklisted |
Most organizations want to be able to at least accept a connection for any source. This corresponds to
smtpd_client_restrictions = permit
If you run separate inbound and outbound Postfix servers for your organization, you could limit connections to the outbound Postfix server by setting
smtpd_client_restrictions = permit_mynetworks, reject
The reject_rbl_client criterion requires additional explanation. Several organizations maintain lists of IP addresses from which spam has been observed to originate. Rejecting connections from IPs in such "real-time black hole" lists (RBLs) can be a extremely effective way to block spam.
Of course, a blacklist is only as good its criteria for including and excluding addresses. The people who maintain such lists are, as you might expect, rather zealous anti-spammers, and have been known to be rather aggressive, blacklisting entire class A (or larger!) network blocks that include a single spam-producing IP, occasionally even blacklisting the IPs of those who have merely been critical of their work. To avoid subjective judgments, some blacklists have turned to automated criteria, but unless the criteria are exquisitely well-chosen, that can lead to more problems than it solves. Here I list, without endorsement, several freely available blackhole lists.
RBL Address | Web Address | Description |
---|---|---|
relays.ordb.org | www.ordb.org | lists open relays; inclusion by testing, removal by request and re-testing |
bl.spamcop.net | www.spamcop.net | lists spammers; inclusion by user reports, removal by expiration |
sbl.spamhaus.org | spamhaus | tracks IP used by a small number of known "spam-gangs" |
spews.relays.osirusoft.com | spews | lists spammers; inclusion and removal by maintainers' judgment |
smtpd_client_restrictions = reject_rbl_client sbl.spamhaus.org,
permit
For a long time, I did not personally use blackhole lists, because I had several times been burned by the RBL server going down. Recently, spam has become such a huge headache that I have begun using RBLs again.
The smtpd_helo_restrictions list makes decisions based on the HELO (or EHLO) greeting provided by the client. Many spammers use fake hostnames or even garbage in their HELO greeting; by rejecting mail from such clients, you may be able to cut down on incoming spam.
reject_invalid_hostname | REJECT if HELO greeting is neither a DN nor an IP |
reject_non_fqdn_hostname | REJECT if HELO greeting is neither a FQDN nor an IP |
reject_unknown_hostname | REJECT if HELO host has no A or MX in DNS |
permit_naked_ip_address | PERMIT if HELO greeting is an IP |
smtpd_helo_restrictions = reject_invalid_hostname, permit
For an overview of which HELO greetings are rejected by which reject_*_hostname restrictions, consult the following table.
blarg! | example | example.com | 192.0.34.72 | |
invalid | X | |||
non_fqdn | X | X | ||
unknown | X | X | ? | X |
Note that reject_unknown_hostname rejects IP addresses as well as fake FQDNs. To allow IP addresses, but still reject fake FQDNs, you can precede reject_unknown_hostnames with permit_naked_ip_address.
smtpd_helo_restrictions = permit_naked_ip_address, reject_invalid_hostname, permit
The smtpd_sender_restrictions list makes decisions based on the MAIL From: address given by the client. To avoid accountability, many spammers use fake From: addresses.
reject_non_fqdn_sender | REJECT if MAIL host is neither a FQDN nor an IP |
reject_unknown_sender_domain | REJECT if MAIL host has no A or MX in DNS |
reject_sender_login_mismatch | REJECT if MAIL username does not match login username |
The reject_unknown_sender_domain restriction is particularly useful, since it rejects mail from an address which cannot be replied to; such mail is almost never from a valid, human correspondent.
smtpd_sender_restrictions = reject_unknown_sender_domain, permit
The smtpd_recipient_restrictions list makes decisions based on the RCPT To: addresses given by the client.
reject_unknown_recipient_domain | REJECT if RCPT host has no A or MX in DNS |
reject_non_fqdn_recipient | REJECT if RCPT host is neither a FQDN nor an IP |
reject_unauth_destination | REJECT unless recipient address is local or in $relay_domains |
permit_auth_destination | PERMIT if recipient address is local or in $relay_domains |
permit_sasl_authenticated | PERMIT if client has authenticated using SASL |
A restriction can be used for its native restriction list or on any restriction list after that. It is often useful to use a restriction on a list later than its native one. For example, the permit_mynetworks restriction is native to the smtpd_client_restrictions list, but we used it on the smtpd_recipient_restrictions list for relay control. A second example would be to use reject_rbl_client (which is also native to the smtpd_client_restrictions list) after permit_sasl_authenticated on the smtpd_recipient_restrictions, so that authorized users are allowed to relay mail even from blacklisted servers.
Any given restriction could block legitimate mail as well as spam. It could also have too small an effect on spam to justify the additional resources (e.g. connecting to a remote DNS server) it requires. Without knowing the rates of true and false positives that a given restriction will generate, it is difficult to justify its adoption. Unfortunately, it is very difficult to make generalizations about the effects of different restrictions; the mail environments in which different organizations operate just differ too greatly.
Fortunately, Postfix allows you to passively test restrictions. If any reject_* restriction is proceeded by warn_if_reject, a REJECT from that restriction results in a log entry instead of a rejection of the mail. For example, if you are considering using reject_unknown_sender_domain, you might want to specify
smtpd_sender_restrictions = warn_if_reject, reject_unknown_sender_domain, permit
Aug 6 10:33:12 smtp postfix/smtpd[23050]: reject_warning:
RCPT from unknown[211.186.53.127]: 450:
Sender address rejected: Domain not found;
from=to=
So far, we have considered criteria that depend exclusively on how a client wishing to send mail presents itself. But it is the content of the mail that, for most users, makes it spam or not. Postfix allows you to reject mail based on regular expression tests of the content of both a mail's headers and a mail's body.
There are a few other directives besides the smtpd_*_restrictions lists that can be usefully employed to fight incoming spam.
To increase mail throughput, some spam-ware doesn't bother to issue a HELO greeting. You can reject mail from clients who don't bother to say HELO.
smtpd_helo_required = yes
Finally, you will probably want to disable the VRFY command, an SMTP language element used to test whether a local addressee exists. The VRFY command is used almost exclusively by the maintainers address lists for bulk mailings to search for valid email addresses.
disable_vrfy_command = yes
Out of the box, Postfix is already sufficiently performant on comodity desktop hardware to handle the email load of a typical medium-sized organization (say 1000 people). You probably need to read this section only if you are configuring an email system for a large organization, or if the activities of your organization require a large volume of automated email (in which case, I hope you are not a spammer).
As any physical chemist knows, the speed of a complex process consisting of a chain of sub-processes is limited by the speed of the slowest sub-process. The same holds true for mail delivery, and if you want to speed up Postfix, you must identify the bottleneck which is limiting mail throughput. Here is a simple checklist of possbile limiting factors:
Postfix consists of about a dozen co-operating applications. The smtpd application accepts incoming mail from the smtp network port. The smtp (without a "d") application contacts other MTAs on the internet to deliver outbound mail. Each of the other applications performs a simple operation on a mail before handing it off to the next one.
By default, Postfix limits the number of concurrent processes of any one application to 50. Since network connections are slow (relative to local interprocess communications), the smtp or smtpd daemons are likely to run up against this limitation first as mail traffic increases. If the number of live smtp or smtpd daemons shown by "ps aux" reaches a significant fraction of this default limit, you should consider increasing the limit. This is done via the default_process_limit directive.
default_process_limit = 100
If you are not running up against the default_process_limit, but still not achieving the mail throughput you want, consider the possibility that you are CPU limited. Use "top" to determine the average CPU utilization of your mail server under peak load. If it is high (say 50% or more), consider purchasing a faster processor.
Before your system is process-limited or CPU-limited, you are likely to be limited by DNS lookup times. Deliverying a mail requires looking up at very least one IP address, and the speed is therefore limited by the latency and bandwith of Postfix's communications with your DNS server. Consider setting up your own caching DNS server right next to your mail server (or, even better, on the mail server machine). Such a simple DNS server is surprisingly simple to set up and maintain, and can have a dramatic impact on DNS lookup times.
If you are not process-limited, CPU-limited, memory-limited, or network-limited, you can still be disk-limited. Postfix maintains a number of queue directories under /var/spool/postfix/. Postfix applications co-operate by picking up mail from one queue directory and, after performing their assiged operation, dropping it into another one. You can go a long way toward increasing Postfix's throughput by speeding access to theses directories. Place /var/spool/postfix on its own hard drive. If you can afford it, use a fast SCSI drive, as SCSI can schedule interleaved requests from multiple processes much better than IDE. If you can't afford a SCSI drive, buy a fast IDE drive and tune it using hdparm to achieve maximum throughput. Whatever drive you use, be sure to mount it with the noatime option to avoid the overhead involved in storing the time of each and every file; Postfix does not use access time stamps.
Imagine you have followed all the steps above, turning your Postfix server into a lean, mean, mailin' machine, and still you cannot reach the level of mail throughput necessary to achieve your goals. (Unless you are a spammer, you are probably going to have to imagine this scenario, because you are unlikely to face it in reality.) If you have a fat enough network pipe to handle several such boxes, you can scale your system up by running several mail servers in parallel.
Since there is no need to maintain state information beyond a single smtp session, you can use a technique called DNS round-robin to great effect. To implement DNS round-robin, assign multiple A entries to the FQDN of your mail hub in DNS.
$ dig smtp.example.com +short
192.0.34.16
192.0.34.17
192.0.34.18
For outbound mail, the scalability of the round-robin technique is limited only by your organization's total network bandwidth. For inbound mail, you may run up agaist the limitations of your IMAP or POP mailstores first.