DBM files for Apache's digest authentication

Summary

Ever tried to use digest authentication using DBM with Apache 2.2? There are no docs, no page on the Internet described how to get it working. But after reading the source I found out how to do it. So this is the page that describes it all.

What DBM?

I found out that there are a couple of DBM formats available. As most of them are obsolete or not really maintained (GNU DBM for example), the consensus seems to be to use Berkeley DB. I used Berkeley 4.5, but I suppose earlier versions work fine as well.

Compiling Apache

You need Apache 2.2 or later. I used Apache 2.2.4.

As I always compile Apache from scratch, I also need something called apr and apr-util. apr-util determines what kind of DBM is supported. As Berkeley DB is the preferred variant, I configured it with:

./configure --prefix=/usr --with-apr=../apr-1.2.8 --with-dbm=db45 --with-berkeley-db

I don't know if --with-berkeley-db was really needed or that --with-dbm=db45 already did the trick, but it didn't seem to harm.

To compile Apache with digest authentication and DBM support, these are the critical parameters for configure:

./configure ... --enable-auth-digest --enable-authn-dbm --enable-authz-dbm

I've no idea what the default DBM type is on FreeBSD. I forced Berkeley DB by installling apr with this command:

cd /usr/ports/devel/apr
make -DAPR_UTIL_WITH_BERKELEY_DB -DAPR_UTIL_WITHOUT_THREADS
make install

The without threads is necessary because my Apache was probably being installed without thread support.

Creating DBM files suitable for digest authentication

Basic authentication is fine when using HTTPS, but not for HTTP as the password is send in almost clear text over the wire. Digest authentication fixes that, but the only documentation available on the Internet shows how to use digest authentication with plain text files. Which won't scale as admitted.

You rather want to use DBM files for example. But the ones used for basic authentication won't work. That's because digest authentication adds one more thing compared with basic authentication: realms. It's like a domain, or a group of users.

But Apache 2.2 doesn't supply any tools to create proper DBM files that includes realms. The supplied htdigest only produces plain text files. dbmmanage and htdbm only produce files that are suitable for basic authentication.

But we can coach them into doing the trick. In this example I will use htdbm as it doesn't need Perl or special libraries.

DBM files consist of key value pairs. The key in basic authentication is just the user name, but for digest files it is user followed by a colon followed by the realm. The value is the password.

This is already sufficient to create DBM files where users can actually be found:

htdbm -Tdb -c users "myname:My Realm"

Or it would be, if htdbm wouldn't complain about the colon, saying it is an invalid character. So you'll need to 'fix' htdbm here. Edit htdbm.c and comment out four lines in the htdbm_valid_username function. The last test in this function is a strchr for a colon, it also has the fprintf with this message. Just comment out this entire if statement and recompile.

Note that I gave the correct DBM format to htdbm. It should be equal to what Apache expects through apr-util. I also specify the -c to create the file, which you obviously only want to do once. htdbm will subsequently prompt for a password and create the users DBM file when this is specified.

However, Apache will not be able to authenticate users with this file as the password will never match. htdbm simply does not create the proper value field, i.e. it does not encode the password properly for digest authentication. What we need to do is to encode the password ourselves, and pass this to htdbm, telling it to store this password as is, not to encode it itself. htdb can accept such passwords if we use the -p parameter. That leaves us with encoding this password.

Digest authentication passwords consist of concatenating the user, realm and password with colon between and encoding this with MD5. Like this:

$ echo -n 'myname:My Realm:password' | md5sum
6ab1403730374a17a0f725b0ab98a0d7 *-

(or md5 on FreeBSD). We need to pass md5sum's checksum to htdbm and this gives us the proper way to create dbm files that can be used for digest authentication:

htdbm -Tdb -c -p users "myname:My Realm" "6ab1403730374a17a0f725b0ab98a0d7"

And that's it. A properly specified DBM file and extremely fast lookups for digest authentication.

.htacess example

To complete this article, here an example of a .htaccess file that specifies digest authentication and DBM files:

AuthType Digest
AuthName "My Realm"
AuthDigestProvider dbm
AuthDBMType db
AuthDBMUserFile /var/db/users

Require valid-user