bl.spamcop.net has minimum.ttl of 0, causing extra queries

[kaos.ocs]

Looking at a customer's DNS traffic (Ubuntu 10.04, bind9 1:9.7.0), I noticed repeated outgoing queries for some names under bl.spamcop.net. The names being queried are not listed in bl.spamcop.net so they get NXDOMAIN. The NXDOMAIN response is not cached, so each mail attempt from these IP addresses results in a new query being sent to spamcop.

Bind9 follows RFC2308 and treats the minimum ttl in the SOA as the cache time for NXDOMAIN responses. The NXDOMAIN response is not cached because the SOA for bl.spamcop.net has a 0 value for minimum ttl. Is this field set to 0 on spamcop by design? It significantly increases the number of DNS requests to spamcop. Some DSNRBL sites have a non-zero value for minimum ttl on the order of 10-60 minutes, so they are not hit with repeated queries for the same name.

Digging through the bind9 source code, there is an undocumented config option, min-ncache-ttl. I set this to 300 seconds and the query rate to spamcop dropped way off. However I don't like overriding values like this without knowing the reason for the original setting.

[Farelf]

Good question, one for the staff (though other users may have some insights). Have taken the liberty of leaving a redirector over in the

SpamCop Discussion > Discussions & Observations > SpamCop Blocklist Help

forum where it might snag some extra attention - noting original post was made right here should that action cause any confusion.

[Wazoo]

Looking at a customer's DNS traffic (Ubuntu 10.04, bind9 1:9.7.0), I noticed repeated outgoing queries for some names under bl.spamcop.net.

Is this customer doing DNS-lookups for something other than handling incoming e-mail?

The SpamCopDNSBL is "very" dynamic .. nothing like other BLs.

[michaelanglo]

Is this customer doing DNS-lookups for something other than handling incoming e-mail?

The SpamCopDNSBL is "very" dynamic .. nothing like other BLs.

Indeed, but some caching is still a good idea since getting hundreds of emails from the same server IP-address in a few minutes is quite likely. Perhaps 30 seconds rather than 300 though.

[kaos.ocs]

Is this customer doing DNS-lookups for something other than handling incoming e-mail?

Spamcop is only used for email. It's a small ISP, about 3M email attempts a day, of which 2.2M are rejected by one RBL or another. All the RBL positives are cached and so are the NXDOMAIN responses from other RBLs; AFAICT it is only the spamcop NXDOMAIN responses that are not being cached. I'm just trying to reduce the network load and delay for legitimate senders.

[sfatula]

Is this customer doing DNS-lookups for something other than handling incoming e-mail?

The SpamCopDNSBL is "very" dynamic .. nothing like other BLs.

I'd also like to see the ttl set to something like 30 seconds. Most anything > 0 would be useful. We use a local caching nameserver, but, we use software called milter-greylist. In that software, you can perform interesting checks. For example, we check if a client is on multiple blacklists. Well, there are a lot of combinations. milter-greylist checks the lists for each comparison. For all other lists, these are cached checks so they basically take 0 seconds. For spamcop, each check for spamcop and some other list actually goes out onto the network since the ttl is set to 0. If it was at least a second or two, you would cut your traffic by quite a bit for all users of milter-greylist (and I am sure other software).

Just a thought, up to you.

[Wazoo]

Sent a note upstream, asking for something more direct. This would probably fall under the need for an 'engineering request' even further upstream, so even if agreed to, it might take some time to implement.

[Wazoo]

From RW;

Thanks. I've sent this to Luci. She can review it and assign it a priority for Engineering to look at.