qmail-verify

On this page: • Introduction • License • Status/Warranty • Documentation • Pre-requisites • Download • Installation • Setup / Configuration • Migrating from realrcptto • Migrating from earlier versions • Compatibility • Gotchas • Security • Programming Notes • Archives

Introduction

This patchset adds a feature to qmail-smtpd's behaviour, enabled by an environment variable: Each recipient specified in the SMTP session (RCPT command) may be verified to see if it is a valid address on the system. A new daemon, qmail-verify is used by qmail-smtpd to determine if addresses are valid; qmail-smtpd communicates with qmail-verify using UDP. qmail-logmsg is a pre-requisite of this package. Note that there are security advisories for qmail-verify (see Security section below for details: If you're not running the latest version of qmail-verify you should update immediately.

This is an update to Paul Jarc's realrcptto patch. The major changes are:

Recipient verification is done by a separate daemon rather than as part of qmail-smtpd.
UDP is used by qmail-smtpd to communicate with qmail-verify, the address verification daemon.
Configuration is different and defaults to off.
qmail-verify is for qmail-smtpd and not qmail-qmtpd: For qmail-qmtpd stick with the realrcptto patch.

License

Paul Jarc's code is used as a basis for this patchset. He raises no objections to this patch. In turn, he has used code from qmail (by Dan Bernstein), who has placed qmail in the public domain (see Dan's page on Information for Distributors).

Status / Warranty

No warranty, express or implied is given - USE THIS SOFTWARE ENTIRELY AT YOUR OWN RISK. You will need to satisfy yourself as to the suitability of this software before deploying it in a production environment.

Documentation

The documentation for this package consists of this web page and the included man pages (the qmail-smtpd man page is an updated version of the original):

qmail-verify(8)
qmail-smtpd(8) - see the 'Recipient Verification' section.

Pre-requisites

This package is for netqmail 1.06 patched with qmail-logmsg versions 1.1, 1.2 or 1.3 (preferred).

Download

The current release, version 1.50, is available in plaintext or gzipped (.gz).

Note that this patch is included in the Debian qmail package / Ubuntu qmail package, derived from the distro's corresponding netqmail source package (links: Debian, Ubuntu).

Installation

For Debian and Ubuntu distros, just install the qmail package for that distro; note that that the fastforward patch mentioned below is included there too.

If instead you're working from source, proceed as follows:

Unpack the patch if you've downloaded a compressed version.
Patch your qmail source: Enter your qmail-1.03 or netqmail-1.06 source directory and type,
patch < path_to_the_patch/qmail-verify_v1.50.patch
which should apply the patch. If this fails, you will need to patch the failed parts by hand. If you're basing your installation on a LWQ style setup, you'd use this patch command just before you compile/build netqmail (make setup check) in section 2.5.5.
[Optional:] If you have an aliases database (using the fastforward package), then there is a patch by Gerrit Pape to make qmail-verify work with this. See discussion of this here (qmail mailing list message from 2010-03-22), or for convenience there's a local copy of his patch here (adjusted to apply cleanly to this version of qmail-verify).
You can now compile qmail and install as normal.
You will need to setup a new 'service' to run qmail-verify, and adjust your qmail-smtpd setup, see the Setup/Configuration section below.

Setup / Configuration

You need to setup a 'service' for qmail-verify.
You need to enable recipient verification in qmail-smtpd.

If you're comfortable with qmail, daemontools etc. you can probably just skim-read this section; if you need a bit more detail, then this section should provide it...

In detail:

Setting up qmail-verify as a 'service' (not necessary if using the Debian/Ubuntu qmail package which sets this up for you in /etc/qmail/qmail-verify):

Assuming you have daemontools installed (as in LWQ), you can create a 'service' as follows (the example paths here follow those of LWQ esp. section 2.8.22, please adjust them for your own installation if necessary):

Create supervise directories for qmail-verify[/log],
mkdir -p /var/qmail/supervise/qmail-verify/log
Change into the supervise/qmail-verify directory,
cd /var/qmail/supervise/qmail-verify
Create the run file,
#!/bin/sh
exec /var/qmail/bin/qmail-verify 2>&1
Alternatively if you wish to listen on a non-default IP address,
#!/bin/sh
exec env LISTEN="192.168.5.5" /var/qmail/bin/qmail-verify 2>&1
Create the log/run file,
#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t /var/log/qmail-verify
Make the run and log/run files executable,
chmod +x run log/run
Create the log directory you've just specified,
mkdir -p /var/log/qmail-verify
chown qmaill /var/log/qmail-verify
Link the supervise/qmail-verify directory into /service:
ln -s /var/qmail/supervise/qmail-verify /service
Check qmail-verify and qmail-verify/log are running:
svstat /service/qmail-verify /service/qmail-verify/log
or with Debian/Ubuntu's qmail packaged qmail-verify,
svstat /etc/qmail/qmail-verify /etc/qmail/qmail-verify/log
which should give a result like,
/service/qmail-verify: up (pid 16646) 100 seconds
/service/qmail-verify/log: up (pid 16831) 100 seconds
If either service shows 'up' for only a couple of seconds, you need to check that you've followed these instructions carefully, you can also get a clue looking at the readproctitle ps entry.

Enabling recipient verification in qmail-smtpd:

Assuming you've setup your qmail-smtpd service with tcpserver and the -x option (as in LWQ, also Debian/Ubuntu packaged version), you just need to update the cdb file referenced by this -x option. The source for this file is typically /etc/tcp.smtp or /etc/qmail/tcp.smtp. Decide on the verification behaviour you prefer (per-recipient or DEFERred) - also see the qmail-smtpd(8) man page - and update the source file accordingly. For example,
127.:allow,RELAYCLIENT=""
192.168.:allow,RELAYCLIENT=""
10.:allow,VERIFY=":"
:allow,VERIFY="DEFER"
which would enable DEFERred validation by default, immediate recipient verification for any 10.x.y.z address, and no recipient verification for 192.168.x.y addresses, nor for the loopback address.
If you've setup qmail-verify on a non-default address (perhaps you're running qmail-verify on a separate machine), you'll also need to specify the address it's listening on - adjust the above to include the server's address, so VERIFY="DEFER,192.168.5.5", for example.
Finally, don't forget to update the cdb file corresponding to the source file you've just edited. If you have a LWQ setup that's,
qmailctl cdb
otherwise (assuming /etc/tcp.smtp, /etc/tcp.smtp.cdb),
tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp
Alternatively if you're not using the -x option to tcpserver you can enable recipient verification for all SMTP connections by setting VERIFY in the environment in which qmail-smtpd is started - for example your startup script might now contain the line
exec env VERIFY="DEFER" /usr/local/bin/tcpserver ...

Migrating from realrcptto

It is fairly straightforward to move between these two approaches to recipient verification.

For qmail-verify to do any recipient verification, the environment variable VERIFY must be setup as described elsewhere on this page.

Recipient verification with realrcptto (the version named qmail-1.03-realrcptto-2006.12.10.patch) is enabled by default. Additionally, QMAILRRTDENYALL can be used to modify its behaviour.

QMAILRRTDENYALL is similar to qmail-verify's VERIFY="DEFER" behaviour: Both cause refusal at the DATA stage.

If you're moving from realrcptto to qmail-verify, you should check your tcpserver cdb source file (if in use) (typically /etc/tcp.smtp):

Change any occurrences of QMAILRRTDENYALL="1" to VERIFY="DEFER"
For lines without QMAILRRTDENYALL="1", add ,VERIFY=":".
If you don't have a default/catch-all line such as :allow, add one (:allow,VERIFY=":").

Now rebuild the corresponding cdb file. Alternatively if you're not using a cdb file for tcpserver, you need to set VERIFY in the tcpserver / qmail-smtpd startup script instead. Remember to rebuild your cdb file having made these changes.

Migrating from earlier versions of qmail-verify

Pre- 1.30 versions of qmail-verify used VERIFY, VERIFYDEFER and VERIFYSVR to specify the required behaviour. From version 1.30 onwards VERIFY has been enhanced to include all the required information: VERIFYDEFER and VERIFYSVR are obsoleted.

Unfortunately the 'meaning' of VERIFY="" is different: This now indicates that verification is disabled, whereas previously this would have enabled verification. Thus when upgrading you need to adjust settings as follows - here is a sample cdb source file that's not yet adjusted:

127.0.0.1:allow

10.:allow,VERIFYDEFER=""

192.168.:allow,VERIFY=""

:allow,VERIFYDEFER="",VERIFYSVR="192.168.1.100:1234"

which would change to,

127.0.0.1:allow

10.:allow,VERIFY="DEFER"

192.168.:allow,VERIFY=":"

:allow,VERIFY="DEFER,192.168.1.100:1234"

Now rebuild the corresponding cdb file. Alternatively if you're not using a cdb file for tcpserver, you need to enable VERIFY in the tcpserver/qmail-smtpd startup script instead. Remember to rebuild your cdb file having made these changes.

Compatibility

Compatibility with qmail, netqmail: The qmail-verify patch v1.50 should apply cleanly to the following:

netqmail-1.05 with qmail-logmsg versions 1.1, 1.2 or 1.3.
netqmail-1.06 with qmail-logmsg versions 1.1, 1.2 or 1.3.

Note: An earlier version of this patch on FreeBSD 4.9 / i386 needed #include "sys/time.h" added to the #include lines in sockbits.c and verifyrcpt.c - this probably applies to this version too. FreeBSD 6.2 was noted as not needing this alteration. Whilst that note is now over a decade old it may be a useful pointer if you're using this patch on a modern FreeBSD.

Compatibility with common patches: Some qmail patches may clash in qmail-smtpd.c, qmail-smtpd.8 or Makefile. Where this occurs (including SMTP AUTH patches, qmail-virusscan and qmail-dk) this will mean some work to finish off the patching by hand: This is however pretty straightforward - alternatively that's bread & butter work for qmail consultants such as the author.

Gotchas

If you're upgrading from an earlier version of qmail-verify the meaning of VERIFY has been changed, and VERIFYDEFER, VERIFYSVR are obsoleted, with their function handled by the value of VERIFY instead - see the upgrading notes.

If you have a qmail system using .qmail-default files for your domains, qmail-verify will consider all addresses in the domain 'valid' (since you have provided explicit delivery instructions for all the domain's addresses by using a .qmail-default file). This includes products such as vpopmail - which may have .qmail-default files like:

| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox

One workaround may be to remove .qmail-default files that are there purely to cause a bounce, instead leaving this task to qmail (and qmail-verify); but check that this doesn't confuse the back-end package.

Sean Newton gives an update here regarding vpopmail & qmail-admin: I just discovered yesterday that qmailadmin is one of the apps that cares a lot about .qmail-default files: It errors out when attempting to administer domains if it doesn't see a .qmail-default file. The fix was a few tweaks to user.c to make it treat the absence of .qmail-default as being equivalent to having read one that said " bounce-no-mailbox\n". The space and the \n are both significant. - find his patch for user.c here - "this will render a qmailadmin-1.2.11 tarball perfectly qmail-verify happy."

Security

Here are some notes relating to security with this patchset, these notes are unlikely to be exhaustive:

There are 2 vulnerabilities in the earlier versions of qmail-verify (fixed in the current version v1.50):

CVE-2020-3811: Not calling control_init(): The default domain if no domain is given in the recipient email address should default to the domain specified in control/me, but without control_init() this doesn't happen, opening up a vulnerability to bypass qmail-verify's recipient address check.
CVE-2020-3812: Calling stat() as root to check for the existence of a .qmail file to determine if an address is valid. A non-root local user of the machine can setup experimental symlinks in place of their normal .qmail file to check for the existence of files that wouldn't normally be visible to them.
At the time of writing those links aren't working, but the 2 vulnerabilities are included in the advisory here.

Please see the advisories linked to above for a good explanation of how these can be exploited (and therefore why you should upgrade to the current version immediately). Both of these issues are fixed in the current version of qmail-verify, discovered and fixed courtesy of Qualys.

It is conceivable that an attacker or spammer could abuse a system patched using this patchset to mount dictionary attacks to determine which email addresses exist on the system. To vary how a qmail-verify patched system responds, you should make sure you understand the difference between immediate and DEFERred recipient verification and set your system appropriately. Immediate verification is likely to be more attractive to an attacker/spammer.

On the other hand, with this patchset you have a greater degree of certainty that messages in the queue for local users are destined for real mailboxes, which also helps to minimise backscatter.

This patchset deliberately splits the functionality of the original realrcptto patch such that the address verification part runs as a separate daemon, qmail-verify (running as root), with qmail-smtpd still able to run as user qmaild. This also means that the changes to qmail-smtpd are minimal.

There is no attempt made to check that the replies to the queries to qmail-verify are genuine: It is conceivable that an attacker might wish to generate spoof UDP replies to tell qmail-smtpd that an address is valid. For this to occur, the attacker would either need to be a 'local' user to generate such 'replies' on the loopback interface (default configuration), or qmail-verify would need to be running on an external interface. In the latter case it should be possible to block the qmail-verify port at a firewall.

All that said however, such an attack is of limited use, since qmail will still bounce the message at delivery time for an invalid recipient.

Programming Notes

Changes to qmail-smtpd.c are minimal, this along with the desire to keep qmail-smtpd running as user qmaild forming the rationale behind updating the realrcptto patch.

Although now fixed in this version 1.50, when I originally wrote qmail-verify I hadn't fully considered the consequences of moving the code into an entirely separate daemon resulting in the 2 vulnerabilities mentioned in Security above: In particular not calling control_init() and not changing the effective UID/GID to the mailbox owner when calling stat().

Although initially using pipes to split the functionality, this was replaced with a UDP client/server setup (with help from John Levine) which was simpler, cleaner and more flexible in how it can be deployed.

In the interests of future extensibility there are a couple of new files with new functions that may be of use to other similar "call-out" type features. These files are sockbits.h, sockbits.c, udpbits.h, udpbits.c, the new functions being query_skt() and connect_udp(). With this structure it should also be more straightforward to replace UDP with (say) Unix domain sockets if that is preferred, either for qmail-verify functionality or for other call-out functions.

qmail-verify.c has some commented-out debug statements, primarily to show - if desired - where in the code qmail-verify decides that an address is valid or not.

qmail-verify.h has ADDR_NOK_TEMP defined, although at present this is not used. This is for possible future use, to indicate that an address is temporarily invalid.

John Levine added code to show the "Controlling user" which is useful for logging esp. from qmail-verify (it can also be enabled in verifyrcpt.c, you'll need to add e.g. eout statements). This doesn't trim trailing 'extension' components to the address, relating to .qmail-example files - evident when using virtual domains.