I've been paying attention lately to the SPF (Sender Policy Framework / Sender Permitted From) initiative. It's an excellent initiative and looks likely to be adopted. However it will be at least a year before it will be of much practical use, even if the world at-large embraces it.

I won't explain how SPF works here. Please see http://spf.pobox.com for a full explanation. The following assumes the reader is familiar with the basics of SPF.

It just occurred to me that the SpamCop webmail system could, with relative ease, adopt a SPF-like filtering capability quickly. I'm finding that despite the SCBL and SpamAssassin, much spam still sneaks through. SpamAssassin 2.70 (SA270) will support SPF (webmail currently uses version 2.63). 99% of the "escaped" spam I see would fail a SPF-like filter test.

I'm hoping that JT would consider implementing a SPF-like filter capability for the webmail system in advance of the SA270 implementation. The SA270 implementation will benefit from (or perhaps even demand something like) this, so the effort would not be wasted.

The suggestion is to have either SC or SC-webmail add a header that contains the transfer relay domain as determined by the first (or gateway) mailhost matched from each user's mailhost list. I suggest something along the lines of "X-SC-Transfer-Domain:". The gateway mailhost is trusted and so any reverse-DNS information provided by it in the "Received: from" header presumably can also be trusted. This relay nominally should be the outbound relay of most valid senders' traffic. To complete the enchantment, the webmail system would be modified to allow e-mail with a mismatched "Return-Path:" (i.e. "envelope sender") and "X-SC-Transfer-Domain:" to be held in a quasi-SPF "purgatory" webmailbox. This mailbox would be separate from the current "held" mailbox because the quasi-SPF filtering mechanism would fail for some valid e-mail. The quantity of mail appearing in SPF purgatory will be small, and will have a relatively high probability of being from a valid sender--thus suggesting a closer and more careful review process.

The key element to understand for those inclined to comment is that SPF revolves around the "envelope sender" indicated by the SMTP "MAIL FROM:" protocol handshake rather than the "From:" header embedded in most messages. Please try to grasp what this is before commenting. See RFC 2821 http://www.apps.ietf.org/rfc/rfc2821.html.

This is not the right metric to worry about. The question is: What percent of non-spam messages would get blocked by such a filter?

In general, though, I support your idea. I'm starting to warm up to the idea of SPF. Until it proves itself, though, the filter would have to be implemented as just a Spam Assassin rule with a relatively low value.

Re-read my suggestion more carefully. A separate "purgatory" held mail box is a key component. This mailbox would accumulate a relatively small number of messages since the standard filters are applied first. False positives will occur, but they are held separately will be easy to identify. Those opting to use this feature will need to review the purgatory mailbox with an order-of-magnitude more care than the normal "held" mail. I bulk-submit my "held" mail with zero review all the time.

The idea is to eliminate the last--very annoying--trickle of escaped spam at the acceptable cost of delaying the rare valid message where the envelope sender does not jive with the transfer relay's domain.

Also. . .

Regarding SA270, my suggestion would not be obsoleted by its implementation
because the SPF support in SA will still depend on wide-spread publication
of SPF DNS "TXT" records. This will take a long time at best.

Ha! I fixed this one myself. I reluctantly setup my own MX a few months ago because my web-host provider kept getting their relays black-listed and this would cause SC to "hold" ALL my mail. Becoming a 'sendmail' expert was never high on my list, but now I've found some added benefit.

It took awhile to decrypt the 'sendmail' documentation, but I finally figured out how to do something I've been wanting for some time: Substitute different envelope recipient addresses based on sender and incoming relay information. The main clue was figuring out that "Ruleset 0" rewrites the "envelope recipient" address. I've created a local ruleset (invoked by ruleset 0) that alters the envelope delivery address (not the "To:" field) if the sending relay is on a whitelist or if the envelope sender matches a different whitelist. The relay list covers the bulk of my high-priority correspondents since they are at large corporations that run their own mail relays and don't have spammers. The envelope sender whitelist covers a handful that have accounts at Yahoo! and like providers that are also sources of spam. Mail that doesn't match the whitelists gets sent to SpamCop to be filtered and then forwarded on to my regular POP accounts. I've reconfigured my screen 'biff' utility to monitor only the preferred sender mailboxes, so now I don't have my concentration broken all day long by idiot spammers. Only important business contacts get immediate attention. The added benefit is that my critical correspondents bypass SpamCop, so I won't get too badly hosed by the occasional SpamCop outages and slow-downs.

For the adventurous, here's the ruleset. Don't bother telling me I should use a mailmap to translate my addresses instead of hard-coding them. I've only got two. Send me the correct rules instead if you think your smart.

I think it would be a good idea, nonetheless, for SpamCop to advertise a SPF record for its own servers. That would allow me to do an include:cesmail.net in my own domain's SPF record. As it is, I don't think I can come up with an SPF record that permits my sending from SpamCop's email system.

Steve

It was my recollection that Julian did this quite a while ago .. now whether that extended to JT's servers .. and in light of the recent relocation ...????? No idea at present, honestly ..

As I recall, Julian added an SPF record for reports.spamcop.net because nobody is supposed to be sending mail with those addresses except for Spamcop's servers. However, spamcop.net by itself is set to "v=spf1 ?all" - which is as it should be (since people send using their spamcop.net addresses from all over the world).

Everybody's mail is different, but benign e-mails that fail SPF are not rare in my e-mail. I get e-mail from complete strangers. For me, the beauty of spamcop filtering is that all these benign e-mails get through, but only a few spams a week get past spamcop's filters.

I cannot have this benign mail going to purgatory. So I would have to increase my spamassassin number (currently set at 2) to compensate for the new spamassassin SPF value, whatever it would be. If I had to increase my spamassassin number, more spam would get through, which would be counterproductive for me.

SPF has some appeal as a concept, and maybe someday there will be a general understanding that everyone has to behave in accordance with it. But in the year 2004, it's too much of a baby/bathwater problem.

You mean SpamAssassin 3.0 (pre-release notes are posted in the Lounge.) I think that the improvement to SA that will benefit SC mail users the most is the URIDNSBL code that has been added. JT could try SA's SPF with a very low contributing score and solicit feedback if and when he implements SA v3.0

I was thinking more that cesmail.net should have an SPF record listing its servers so that those of us who use the webmail system to send mail on behalf of our own domains would be able to have an include:cesmail.net in our own SPF record. It doesn't.