OpenDKIM Key Retrieval Failed

I set up OpenDKIM on my mailserver years ago, but while I got it working for signing just fine, I could never get it working for verification. Whenever I'd receive a message signed with DKIM, I'd see an error message like this in my mailserver logs:

May  4 01:20:20 mail opendkim[24874]: 7EE5982132: key retrieval failed (s=20161025, d=gmail.com): '20161025._domainkey.gmail.com' query timed out

When I tried dig on the DNS record listed in the logs, however, I was able to retrieve it just fine:

dig TXT 20161025._domainkey.gmail.com

Recently I set aside some time to "dig" into it further. I found I could use the opendkim-testkey command to at least reproduce the issue (instead of having to keep sending test emails from other accounts to myself). For example, the following command tries to retrieve the DKIM key from the 20161025._domainkey.gmail.com DNS TXT record (20161025 is the DKIM selector and gmail.com is the signing domain):

opendkim-testkey -s 20161025 -d gmail.com

This command gave me the same "query timed out" error message that I saw in my logs. Through a lot of trial and error, I figured out that I could avoid the error by setting the Nameservers property in my /etc/opendkim.conf file to an external DNS server (any external one will do), and restarting the OpenDKIM daemon. I've been running my mailserver on an Ubuntu EC2 instance, and apparently OpenDKIM does not like something about the combination of the Ubuntu DNS resolver and the internal EC2 DNS servers.

So, I added this line to my /etc/opendkim.conf, restarted the OpenDKIM daemon, and now I no longer see the "key retrieval failed" error message in my logs (instead I get a nice Authentication-Results header added by OpenDKIM to my incoming mail)!:

Nameservers 1.1.1.1

1.1.1.1 is Cloudflare's DNS servers — but any external DNS servers should work. One thing to keep in mind with using external DNS servers is that if your network has a stateless firewall, you need to allow inbound access to UDP in the "ephemeral" port range. If you're using EC2's Network ACLs (and not just using the defaults), this means adding a rule like the following to the ACL for the subnet in which your mailserver lives (32768-61000 is Ubuntu's ephemeral port range):

Rule #: [any number lower than your DENY rules]
Type: Custom UDP Rule
Protocol: UDP (17)
Port Range: 32768-61000
Source: 1.1.1.1/32
Allow/Deny: ALLOW

On Donner and Blitzen

The first company I worked for was pretty much a Windows-only shop. They used Exchange for email, VSS for version control, and Visual Studio as an IDE. This was the first time I had ever used Windows, and I disliked it (although not as badly as I thought I would) — but I didn't really have a choice. Since 2004, though, I've been fortunate enough to be able to use Linux on my primary dev box at work. My current company (and all the ones I've ever worked for) still uses Exchange, so I've struggled with how to deal with this over the years.

Now Dasher! Now Dancer!

When I first started using Windows, I didn't know the difference between Outlook and Outlook Express (apparently the former is an Exchange client and the other is a generic SMTP/POP client), and so I used Outlook Express (because I wanted my mail faster, of course). After a few months I finally figured out why I couldn't "accept" meeting requests: I needed to use Outlook for that. So I switched to Outlook proper; and even after I moved my primary dev box to Linux, I still kept a Windows machine on the local Windows domain to keep using Outlook (and also to test IE, of course).

Now Prancer and Vixen!

But switching back and forth between my Windows and Linux boxes got old pretty quick. First I tried Eudora as a SMTP/POP client (which I had used on the old Mac OS before OS X). That was okay, but POP is pretty primitive and usually doesn't work well (it's hard to keep the local read/deleted states in sync with the server, so you usually just end up downloading everything locally and deleting it on the server — which means you better have a good local backup story).

On Comet! On Cupid!

Then I tried Evolution as an IMAP client, which works (at least with Exchange) much better than POP. You can keep mail in sync on the server, and even use Exchange's server-side folder structure. The two big things missing still, though, are contacts and meetings. Evolution is supposed to be able to integrate more tightly with Exchange (possibly including these two features) via Exchange's web interface, but I've never been able to get that to work. So I just used Exchange's web interface whenever I wanted to lookup a person in Exchange, or to schedule/respond-to a meeting.

On Donner

Eventually I got restless, and decided to try Thunderbird. I can't say that it's really any better or worse than Evolution, just somewhat different. It's what I use now, but it still doesn't have contacts and meetings (although for a while I got it little bit of contact integration by syncing it with the corporate LDAP server). But then one day I found Lightning (and you're welcome for taking so long to get to the point of this post).

And Blitzen! (aka Finally, the Point of this Post)

Lightning, an extension for Thunderbird, allows just enough integration with Exchange's meeting functionality to be really really useful to me. Lightning is mainly a calendaring extension, but it's also able to parse Exchange's meeting-request emails — it displays a button right in the email to add the meeting-request to your Lightning calendar. So now I finally have a way to organize my meeting schedule (other than to highlight the meeting-request emails in my inbox — plus in the past Exchange unhelpfully moved these emails to a separate Calendar folder if I used the Exchange web interface to accept the meeting).

Here are some pics of the Lightning "Today Pane" (displayed on the right-side of my normal email listings), and the full Lightning "Calendar" interface (which is pretty much exactly what you'd expect and need from a calendering UI). My apologies for the super-gay Thunderbird persona (I can't seem to login to the Personas website in Thunderbird, so for now I'm stuck with picking one of the "popular" personas).

Lightning "Today Pane"

Lightning "Calendar"

On 64-Bit Linux

The one gotcha with this (and kind of my motivation for writing this post) is that the version of Lightning from addons.mozilla.org doesn't work on 64-bit Linux. Fortunately, Ubuntu has this documented and taken care of on their ThunderbirdLightning Community Documentation page. To make it work, I just had to download the 64-bit lightning.xpi, and install it manually (via the Install... button in the Extensions tab of the Add-ons dialog, which you can get at via the Tools > Add-ons menu item). Now dash away, dash away, dash away all!