qmail-verify (OLD VERSION - v1.32)

This version of qmail-verify is the subject of a security advisory: Please upgrade to the latest version instead

On this page: • Introduction • License • Status/Warranty • Documentation • Pre-requisites • Download • Installation • Setup / Configuration • Migrating from realrcptto • Migrating from earlier versions • Compatibility • Gotchas • Security • Programming Notes • Archives

Introduction

This patchset adds a feature to qmail-smtpd's behaviour, enabled by an environment variable: Each recipient specified in the SMTP session (RCPT command) may be verified to see if it is a valid address on the system. A new daemon, qmail-verify is used by qmail-smtpd to determine if addresses are valid; qmail-smtpd communicates with qmail-verify using UDP. qmail-logmsg is a pre-requisite of this package.

This is an update to Paul Jarc's realrcptto patch. The major changes are:

Recipient verification is done by a separate daemon rather than as part of qmail-smtpd.
UDP is used by qmail-smtpd to communicate with qmail-verify, the address verification daemon.
Enabling recipient verification is different, and defaults to off.
No code has been written for qmail-qmtpd to use qmail-verify, and none is planned: If you use qmail-qmtpd and need address verification, stick with the realrcptto patch.

License

I have used Paul Jarc's code as a basis for this patchset. He raises no objections to this patch. In turn, he has used code from qmail (by Dan Bernstein), who has placed qmail in the public domain (see Dan's page on Information for Distributors).

Status / Warranty

No warranty, express or implied is given - USE THIS SOFTWARE ENTIRELY AT YOUR OWN RISK. You will need to satisfy yourself as to the suitability of this software before deploying it in a production environment.

Documentation

The documentation for this package consists of this web page and the included man pages (the qmail-smtpd man page is an updated version of the original):

qmail-verify(8)
qmail-smtpd(8) - see the 'Recipient Verification' section.

Pre-requisites

This package is for netqmail 1.06 patched with qmail-logmsg versions 1.1, 1.2 or 1.3 (preferred).

Download

[NB old version] Version 1.32, is available in plaintext or gzipped (.gz), but note that this version is subject to a security advisory: Please use the latest version instead.

Installation

Proceed as follows:

Unpack the patch if you've downloaded a compressed version.
Patch your qmail source: Enter your qmail-1.03 or netqmail-1.06 source directory and type,
patch < path_to_the_patch/qmail-verify_v1.32.patch
which should apply the patch. If this fails, you will need to patch the failed parts by hand. If you're basing your installation on a Life with qmail style setup, you'd use this patch command just before you compile/build netqmail (make setup check) in section 2.5.5.
You can now compile qmail and install as normal.
You will need to setup a new 'service' to run qmail-verify, and adjust your qmail-smtpd setup, see the Setup/Configuration section below.
I'd appreciate your report on whether you could compile qmail successfully with this patch so that I can update the Compatibility section below - please email qv-compat@acrconsulting.co.uk with details of your system (esp. OS & CPU) - just make systype; cat systype from within your qmail source directory is enough if you're in a hurry!

Setup / Configuration

You need to setup a 'service' for qmail-verify.
You need to enable recipient verification in qmail-smtpd.

If you're comfortable with qmail, daemontools etc. you can probably just skim-read this section; if you need a bit more detail, then this section should provide it...

In detail:

Setting up qmail-verify as a 'service':

You need to run qmail-verify - it runs as a daemon. It normally listens for UDP requests on the loopback address, 127.0.0.1. For testing purposes you can run it from the command line (it will generate output when UDP requests come in).
Assuming you have daemontools installed (as in LWQ), you can create a 'service' as follows (the example paths here follow those of LWQ esp. section 2.8.22, please adjust them for your own installation if necessary):

Create supervise directories for qmail-verify[/log],
mkdir -p /var/qmail/supervise/qmail-verify/log
Change into the supervise/qmail-verify directory,
cd /var/qmail/supervise/qmail-verify
Create the run file,
#!/bin/sh
exec /var/qmail/bin/qmail-verify 2>&1
Alternatively if you wish to listen on a non-default IP address,
#!/bin/sh
exec env LISTEN="192.168.5.5" /var/qmail/bin/qmail-verify 2>&1
Create the log/run file,
#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t /var/log/qmail-verify
Make the run and log/run files executable,
chmod +x run log/run
Create the log directory you've just specified,
mkdir -p /var/log/qmail-verify
chown qmaill /var/log/qmail-verify
Link the supervise/qmail-verify directory into /service:
ln -s /var/qmail/supervise/qmail-verify /service
Check qmail-verify and qmail-verify/log are running:
svstat /service/qmail-verify /service/qmail-verify/log
which should give a result like,
/service/qmail-verify: up (pid 16646) 100 seconds
/service/qmail-verify/log: up (pid 16831) 100 seconds
If either service shows 'up' for only a couple of seconds, you need to check that you've followed these instructions carefully, you can also get a clue looking at the readproctitle ps entry.

Enabling recipient verification in qmail-smtpd:

Assuming you've setup your qmail-smtpd service with tcpserver and the -x option (as in LWQ), you just need to update the cdb file referenced by this -x option. The source for this file is typically /etc/tcp.smtp. Decide on the verification behaviour you prefer (per-recipient or DEFERred) - also see the qmail-smtpd(8) man page - and update the source file accordingly. For example,
127.:allow,RELAYCLIENT=""
192.168.:allow,RELAYCLIENT=""
10.:allow,VERIFY=":"
:allow,VERIFY="DEFER"
which would enable DEFERred validation by default, immediate recipient verification for any 10.x.y.z address, and no recipient verification for 192.168.x.y addresses, nor for the loopback address.
If you've setup qmail-verify on a non-default address (perhaps you're running qmail-verify on a separate machine), you'll also need to specify the address it's listening on - adjust the above to include the server's address, so VERIFY="DEFER,192.168.5.5", for example.
Finally, don't forget to update the cdb file corresponding to the source file you've just edited. If you have a LWQ setup that's,
qmailctl cdb
otherwise (assuming /etc/tcp.smtp, /etc/tcp.smtp.cdb),
tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp
Alternatively if you're not using the -x option to tcpserver you can enable recipient verification for all SMTP connections by setting VERIFY in the environment in which qmail-smtpd is started - for example your startup script might now contain the line
exec env VERIFY="DEFER" /usr/local/bin/tcpserver ...

Migrating from realrcptto

It is fairly straightforward to move between these two approaches to recipient verification.

For qmail-verify to do any recipient verification, the environment variable VERIFY must be setup as described elsewhere on this page.

Recipient verification with realrcptto (the version named qmail-1.03-realrcptto-2006.12.10.patch) is enabled by default. Additionally, QMAILRRTDENYALL can be used to modify its behaviour.

QMAILRRTDENYALL is similar to qmail-verify's VERIFY="DEFER" behaviour: Both cause refusal at the DATA stage.

If you're moving from realrcptto to qmail-verify, you should check your tcpserver cdb source file (if in use) (typically /etc/tcp.smtp):

Change any occurrences of QMAILRRTDENYALL="1" to VERIFY="DEFER"
For lines without QMAILRRTDENYALL="1", add ,VERIFY=":".
If you don't have a default/catch-all line such as :allow, add one (:allow,VERIFY=":").

Now rebuild the corresponding cdb file. Alternatively if you're not using a cdb file for tcpserver, you need to set VERIFY in the tcpserver / qmail-smtpd startup script instead. Remember to rebuild your cdb file having made these changes.

Migrating from earlier versions of qmail-verify

Earlier (pre. 1.30) versions of qmail-verify used VERIFY, VERIFYDEFER and VERIFYSVR to specify the required behaviour. As of version 1.30 VERIFY has been enhanced to include all the required information: VERIFYDEFER and VERIFYSVR are obsoleted.

Unfortunately the 'meaning' of VERIFY="" is different: This now indicates that verification is disabled, whereas previously this would have enabled verification. Thus when upgrading you need to adjust settings as follows - here is a sample cdb source file that's not yet adjusted:

127.0.0.1:allow

10.:allow,VERIFYDEFER=""

192.168.:allow,VERIFY=""

:allow,VERIFYDEFER="",VERIFYSVR="192.168.1.100:1234"

which would change to,

127.0.0.1:allow

10.:allow,VERIFY="DEFER"

192.168.:allow,VERIFY=":"

:allow,VERIFY="DEFER,192.168.1.100:1234"

Now rebuild the corresponding cdb file. Alternatively if you're not using a cdb file for tcpserver, you need to enable VERIFY in the tcpserver/qmail-smtpd startup script instead. Remember to rebuild your cdb file having made these changes.

Compatibility

Please let me know if you successfully install qmail-verify on systems not listed below (see the end of the installation section on this page).

Compatibility with qmail, netqmail: The qmail-verify patch v1.32 should apply cleanly to the following:

netqmail-1.05 with qmail-logmsg versions 1.1, 1.2 or 1.3.
netqmail-1.06 with qmail-logmsg versions 1.1, 1.2 or 1.3.

Compatibility with Unix / Linux: netqmail 1.05 / netqmail 1.06 with the qmail-verify patch v1.32 has been reported to compile successfully on the following:

Linux:

Centos 5.3 / i386 (Systype: linux-2.6.18-128.1.10.el5pae-:i386-:-:ppro-:-)
Fedora 8 / i386 (systype: linux-2.6.26.8-57.fc8-:i386-:-:ppro-:-)
Ubuntu 8.04.1 / i386 (systype: linux-2.6.24-19-generic-:i386-:-:ppro-:-)
Ubuntu 9.04 / i386 (systype: linux-2.6.28-15-generic-:i386-:-:ppro-:-)
Debian 6.0.2 / amd64 (systype: linux-2.6.32-5-amd64-:-:-:x86_64-:-)

Unix:

OpenBSD 4.0 / Ultrasparc (systype: openbsd-4.0-custom#1-:openbsd.sparc64-:-:sparc64-:-)
Note: the previous version of this patch on FreeBSD 4.9 / i386 needed #include "sys/time.h" added to the #include lines in sockbits.c and verifyrcpt.c - this probably applies to this version too. FreeBSD 6.2 was noted as not needing this alteration. No reports for either of these FreeBSDs has been received yet for this version of the patch.

Compatibility with common patches: In trying a few typical qmail patches, I note that they may clash in qmail-smtpd.c, qmail-smtpd.8 or Makefile. In patches I looked at (including SMTP AUTH patches, qmail-virusscan and qmail-dk) this will mean some work to finish off the patching by hand: This is however pretty straightforward - alternatively that's bread & butter work for qmail consultants such as the author.

Gotchas

If you're upgrading from an earlier version of qmail-verify the meaning of VERIFY has been changed, and VERIFYDEFER, VERIFYSVR are obsoleted, with their function handled by the value of VERIFY instead - see the upgrading notes.

If you have a qmail system using .qmail-default files for your domains, qmail-verify will consider all addresses in the domain 'valid' (since you have provided explicit delivery instructions for all the domain's addresses by using a .qmail-default file). This includes products such as vpopmail - which may have .qmail-default files like:

| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox

One workaround may be to remove .qmail-default files that are there purely to cause a bounce, instead leaving this task to qmail (and qmail-verify); but check that this doesn't confuse the back-end package.

Sean Newton gives an update here regarding vpopmail & qmail-admin: I just discovered yesterday that qmailadmin is one of the apps that cares a lot about .qmail-default files: It errors out when attempting to administer domains if it doesn't see a .qmail-default file. The fix was a few tweaks to user.c to make it treat the absence of .qmail-default as being equivalent to having read one that said " bounce-no-mailbox\n". The space and the \n are both significant. - find his patch for user.c here - "this will render a qmailadmin-1.2.11 tarball perfectly qmail-verify happy."

Security

Here are some notes relating to security with this patchset, these notes are unlikely to be exhaustive:

It is conceivable that an attacker or spammer could abuse a system patched using this patchset to mount dictionary attacks. To vary how a qmail-verify patched system responds, you should make sure you understand the difference between immediate and DEFERred recipient verification and set your system appropriately. Immediate verification is likely to be more attractive to an attacker/spammer.

On the other hand, with this patchset you have a greater degree of certainty that messages in the queue for local users are destined for real mailboxes.

This patchset deliberately splits the functionality of the original realrcptto patch such that the address verification part runs as a separate daemon, qmail-verify (running as root), with qmail-smtpd still able to run as user qmaild. This also means that the changes to qmail-smtpd are minimal.

There is no attempt made to check that the replies to the queries to qmail-verify are genuine: It is conceivable that an attacker might wish to generate spoof UDP replies to tell qmail-smtpd that an address is valid. For this to occur, the attacker would either need to be a 'local' user to generate such 'replies' on the loopback interface (default configuration), or qmail-verify would need to be running on an external interface. In the latter case it should be possible to block the qmail-verify port at a firewall.

All that said however, such an attack is of limited use, since qmail will still bounce the message at delivery time for an invalid recipient.

Programming Notes

I've tried to ensure that the changes to qmail-smtpd.c are minimal, this along with the desire to keep qmail-smtpd running as user qmaild forming the rationale behind updating the realrcptto patch.

Although I initially used pipes to split the functionality, I replaced this with a UDP client/server setup (with help from John Levine) which was simpler, cleaner and more flexible in how it can be deployed.

In the interests of future extensibility I've created a couple of new files with new functions that may be of use to other similar "call-out" type features. These files are sockbits.h, sockbits.c, udpbits.h, udpbits.c, the new functions being query_skt() and connect_udp(). With this structure it should also be more straightforward to replace UDP with (say) Unix domain sockets if that is preferred, either for qmail-verify functionality or for other call-out functions.

qmail-verify.c has some commented-out debug statements, primarily to show - if desired - where in the code qmail-verify decides that an address is valid or not.

qmail-verify.h has ADDR_NOK_TEMP defined, although at present this is not used. This is for possible future use, to indicate that an address is temporarily invalid.

John Levine added code to show the "Controlling user" which is useful for logging esp. from qmail-verify (it can also be enabled in verifyrcpt.c, you'll need to add e.g. eout statements). This doesn't trim trailing 'extension' components to the address, relating to .qmail-abc files - evident when using virtualdomains.