Jump to content
Sign in to follow this  
DavidT

Update to blacklist options needed

Recommended Posts

While this may be seen as a "feature request," it doesn't involve changes to the reporting "codebase or user interface," so I hope it can be allowed to remain in the SpamCop Email System & Accounts, rather than sent to the black hole of the "Feature Requests" zone. :P

Much of my email is double filtered, running through a Barracuda box before it gets to my SpamCop email account, so on false negatives, I often take a look at what the Barracuda thought about the item that the SpamCop filters didn't divert to my Held mail. I'm seeing a significant amount of items in which the Barracuda is reporting "RBL: Received via a relay in Spamhaus PBL" and then "[(IP #) listed in zen.spamhaus.org]." I note that when Merlyn supplied detailed BL lookups on given IPs, that "zen.spamhaus.org" is sometimes mentioned, such as in this post:

http://forum.spamcop.net/forums/index.php?...ost&p=57478

where you'll see this:

ZEN Spamhaus combined SBL, XBL and PBL - replaces SBLXBL: zen.spamhaus.org -> 127.0.0.11

In our SC email blacklist options, we have both the SBL and the XBL, but we're missing the PBL, which means that spam is getting through to us unnecessarily (and I've got items in my inbox that prove this). So, I'm requesting that either the PBL get added, or that the current SBL and XBL get replaced by the all-encompassing ZEN. Here's the page at Spamhaus that describes ZEN:

http://www.spamhaus.org/zen/

Using ZEN instead of both SBL and XBL should speed up the analysis at the SC email servers because only a single query, instead of two, would need to happen. So adding the PBL as a third option would be extra work for the server, while replacing SBL/XBL with ZEN would probably reduce the server load, and due to the quality of the Spamhaus BLs, would not lead to false positives. I just searched the headers of over 3,000 "good" messages in my mail folders and came up with only one false positive (and it was a rather spammy sender).

There have been several topics recently about SC email customers seeing more spam slipping through to their inboxes (partly due to the weakness of our un-trained SA implementation, IMO), and this would help. I mentioned it in this recent topic:

http://forum.spamcop.net/forums/index.php?showtopic=8865

but I think the comment was missed, which is why I'm starting a new topic dedicated to the subject.

I'm hoping that Trevor drops by soon and will consider this request. I'm also hoping that other SC email customers will chime in with "me, too" responses, because wouldn't we all like to see fewer false negatives making it to our inboxes? :)

DT

Edited by DavidT

Share this post


Link to post
Share on other sites

I'm hoping that Trevor drops by soon and will consider this request. I'm also hoping that other SC email customers will chime in with "me, too" responses, because wouldn't we all like to see fewer false negatives making it to our inboxes? :)

DT

I don't think Trevor is regularly "dropping into" the Forums. The best bet would be to direct him here with an email.

Also, I disagree with your initial assessment. The New Features should be about discussing ANY change to the systems, including this one.

That being said, I fully support this request, and any other BL's that would appear to help block more spam to the Inbox.

Share this post


Link to post
Share on other sites
<snip>

Also, I disagree with your initial assessment. The New Features should be about discussing ANY change to the systems, including this one.

<snip>

...Not only that, but my sense is that the reason that the New Features forum seems to be a black hole is not because the entries are in that forum but because they are requests that TPTB do not consider high enough priority to respond to, so placing what really is a new feature request into a different forum (such as the SpamCop Email System & Accounts) will not really help.

Share this post


Link to post
Share on other sites
placing what really is a new feature request into a different forum (such as the SpamCop Email System & Accounts) will not really help.

Not sure I buy that. To convince me otherwise, I need to see an example of something posted initially into "New Feature Request" that resulted in the request being granted and implemented.

Besides, the name of that forum is "New Feature Request," and I'm not requesting a new feature, but rather a fairly straightforward modification of an existing feature. And the feature only applies to SpamCop Email accounts. There's also precendence for certain feature requests to stay out out of "New Feature Requests," specifically: "(NOTE: MailHost stuff should stay in the MailHost Forum)"

I'll send a note through the official channel, although Trevor does make frequent visits...he was here this morning, after I started this topic.

DT

Share this post


Link to post
Share on other sites
Not sure I buy that. To convince me otherwise, I need to see an example of something posted initially into "New Feature Request" that resulted in the request being granted and implemented.
...IMHO, that's not worth trying to research and may not be demonstrable. My point is that IMHO a request is just as likely to be ignored in any other forum as it is in the New Feature Request forum.
Besides, the name of that forum is "New Feature Request," and I'm not requesting a new feature, but rather a fairly straightforward modification of an existing feature.
...IMHO that's taking the "New" part of the name of the forum too literally. It could be considered as a "new" feature in the sense that you are asking for something that does not currently exist -- the addition of a blacklist.
<snip>

There's also precendence for certain feature requests to stay out out of "New Feature Requests," specifically: "(NOTE: MailHost stuff should stay in the MailHost Forum)"

<snip>

...That is only in terms of what is on topic / off topic. I wouldn't claim that your request is off-topic, merely expressing my opinion (to further StevenUnderwood's) that it makes little or no difference whether you had posted it to the New Feature Request forum or the SpamCop Email System & Accounts forum in respect of its likelihood of being adopted.

Share this post


Link to post
Share on other sites
Not sure I buy that. To convince me otherwise, I need to see an example of something posted initially into "New Feature Request" that resulted in the request being granted and implemented.

Look for the [Resolved] tag ... of course, noting that the majority are those dealing with this Forum .... but there a few others. Yes, I'll also admit that resolutions, even 'working' the issue, was actually based on contacts, dialog, etc. made outside of this Forum section in a number of these, but .... [Resolved] they were.

I'll send a note through the official channel, although Trevor does make frequent visits...he was here this morning, after I started this topic.

Lots of looking and searching .... know not what for ....

Share this post


Link to post
Share on other sites
Look for the [Resolved] tag ... of course, noting that the majority are those dealing with this Forum .... but there a few others.

The key word there is "few." The most recent "Resolved" issue that involved getting some action from JT was exactly a year ago, and I was instrumental in that one....and the primary discussion took place in the Email forum...not the New Feature Requests. The issue wasn't even fully resolved, in that we've still got the three "*.blackholes.us" BL options, even though that site hasn't shown any sign of life in over two years!

The next "Resolved" item (going back in time) that doesn't involve Forum fixes or confused users who simply needed some instructions was from February 2004, also involving the BL list for Email customers. So, the resolutions that involve the owners/admins throwing a bone to we the users are unfortunately few and far between.

DT

Share this post


Link to post
Share on other sites
In our SC email blacklist options, we have both the SBL and the XBL, but we're missing the PBL, which means that spam is getting through to us unnecessarily (and I've got items in my inbox that prove this). So, I'm requesting that either the PBL get added, or that the current SBL and XBL get replaced by the all-encompassing ZEN. Here's the page at Spamhaus that describes ZEN:

http://www.spamhaus.org/zen/

Despite all the forum policies and politics about where this topic should be placed (which is really irrelevant, IMO), I completely agree w/ David that adding SH Zen would be a great benefit to the 0SpamCop email. There are a few other BLs I can think of I would like to add, but I would definitely put Zen at #1. That would also allow the removal of the SpamCop SBL, XBL and technically the CBL since it pulls it's info from the XBL.

In fact, SpamHaus themselves, at the website linked above, say:

In most cases, zen.spamhaus.org replaces sbl-xbl.spamhaus.org. If you are currently using sbl-xbl.spamhaus.org you should now replace 'sbl-xbl.spamhaus.org' with 'zen.spamhaus.org'.

zen.spamhaus.org should now be the only spamhaus.org DNSBL in your configuration. You should not use ZEN together with other Spamhaus blocklists, or with blocklists already included in our zones (such as the CBL) or you will simply be wasting DNS queries and slowing your mail queue.

Edited by jongrose

Share this post


Link to post
Share on other sites

A little data: I just did a little search of my Held mail this morning, and 65% of the messages come from IPs on the SpamHaus Zen list.

In any case, I'd like to ask why SC Email is still using three "*.blackholes.us" options on the filtering blacklists? My research tells me that the owner/maintainer hasn't touched those in over two years (!), so they are horribly out of date. His website was last updated over two years ago, and before that, he had also disappeared out of NANAE, where he had apparently been a denizen. He had some big server/access problems years ago, then some problems with the Miami/Dade police department because their email was being blocked due to one of his ISP-specific (Bellsouth) blacklists, and then he simply disappeared and nobody has heard from him.

Here are two links to archived forum posts mentioning his disappearance (including his phone being disconnected), and then someone reporting that he was unresponsive/unreachable in 2006:

http://www.webservertalk.com/archive154-2004-9-393624.html

http://www.webservertalk.com/archive154-2006-5-1533581.html

(I'm not related to the person who made the second post, despite the similarity of initials)

DT

Edited by DavidT

Share this post


Link to post
Share on other sites

It's been on the list of stuff to do for a long time, but that particular list works differently from all of the others and isn't compatible with our software. Since there has been so much interest in it, though, I started the code modifications today. No exact time frame, but probably a couple of days until it is up. We're going to replace the two existing Spamhaus entries with one Zen entry, and move everyone over to that one.

-Trevor

Share this post


Link to post
Share on other sites
It's been on the list of stuff to do for a long time, but that particular list works differently from all of the others and isn't compatible with our software.
...did anyone ever post that here before?

Since there has been so much interest in it, though, I started the code modifications today. No exact time frame, but probably a couple of days until it is up. We're going to replace the two existing Spamhaus entries with one Zen entry, and move everyone over to that one.

Very good! No big hurry...just glad there's some movement. THANKS! :D

While you're at it...you really ought to look into the efficacy of keeping those apparently outdated "*.blackholes.us" options online (I was adding that information to my post above as you were posting this good news). AFAICT, the data behind those lists are *extremely* stale and probably no longer accurate...unless someone can prove that Matthew J. Evans is still updating his site/lists?

Thanks again,

DT

Share this post


Link to post
Share on other sites
...unless someone can prove that Matthew J. Evans is still updating his site/lists?

Thanks again,

DT

Seems slightly odd that the website would still be up if no one is maintaining it. I did look at the WHOIS data from www.whois.us

Domain Name BLACKHOLES.US

Domain ID D2072939-US

Sponsoring Registrar TUCOWS INC.

Domain Status ok

Registrant ID TUWDT6SG3RG3CCLN

Registrant Name Matthew Evans

Registrant Organization Shakha Enterprises

Registrant Address1 PO Box 1017

Registrant City Chimayo

Registrant State/Province NM

Registrant Postal Code 87522

Registrant Country United States

Registrant Country Code US

Registrant Phone Number +1.5057533825

Registrant Email matthew[at]blackholes.us

Registrant Application Purpose P3

Registrant Nexus Category C11

Administrative Contact ID TUWDT6SG3RG3CCLN

Administrative Contact Name Matthew Evans

Administrative Contact Organization Shakha Enterprises

Administrative Contact Address1 PO Box 1017

Administrative Contact City Chimayo

Administrative Contact State/Province NM

Administrative Contact Postal Code 87522

Administrative Contact Country United States

Administrative Contact Country Code US

Administrative Contact Phone Number +1.5057533825

Administrative Contact Email matthew[at]blackholes.us

Name Server SCARLATTI.SHAKHA.COM

Name Server NS5.BLACKHOLES.US

Created by Registrar TUCOWS INC.

Last Updated by Registrar TUCOWS INC.

Domain Registration Date Fri Apr 26 04:21:01 GMT 2002

Domain Expiration Date Fri Apr 25 23:59:59 GMT 2008

Domain Last Updated Date Sun Apr 15 19:46:16 GMT 2007

I checked the email address listed there, and it is still valid. No mention of it on dnsbl.com

Share this post


Link to post
Share on other sites
Seems slightly odd that the website would still be up if no one is maintaining it.
Yes, odd, but not impossible. There are lots of old sites still online for which hosting payments may not have been a month-to-month issue, or perhaps are set up with "autopay" renewal options, and so, as long as the domain name gets renewed (which can happen automatically, as long as the credit card is still good), the sites live on.

I did look at the WHOIS data from www.whois.us

I checked the email address listed there, and it is still valid. No mention of it on dnsbl.com

How did you check the email address? Unless you actually receive a response from him, he might be blackholing all incoming messages.

DT

Share this post


Link to post
Share on other sites
...How did you check the email address? Unless you actually receive a response from him, he might be blackholing all incoming messages.
True, actual msg and actual response are the only sure test but short of that (my own favorite verifier)
[Contacting scarlatti.shakha.com [216.243.118.34]...]

[Connected]

220 scarlatti.shakha.com ESMTP Sendmail 8.13.3/8.13.3; Wed, 24 Oct 2007 11:39:58 -0600 (MDT)

EHLO hexillion.com

250-scarlatti.shakha.com Hello mail.webpal.info [70.84.211.98] (may be forged), pleased to meet you

250-ENHANCEDSTATUSCODES

250-PIPELINING

250-EXPN

250-VERB

250-8BITMIME

250-SIZE

250-DSN

250-ETRN

250-DELIVERBY

250 HELP

NOOP *** See <http://www.hexillion.com/MailAdmin/> for an explanation of this session

250 2.0.0 OK

NOOP *** HexValidEmail COM 1.4.12 <5c31a8fa73d35685c3baa1e0430da151bdc52a85>

250 2.0.0 OK

RSET

250 2.0.0 Reset state

MAIL FROM:<HexValidEmail[at]hexillion.com>

250 2.1.0 <HexValidEmail[at]hexillion.com>... Sender ok

RCPT TO:<hextest13F8[at]blackholes.us>

550 5.1.1 <hextest13F8[at]blackholes.us>... User unknown

RCPT TO:<matthew[at]blackholes.us>

250 2.1.5 <matthew[at]blackholes.us>... Recipient ok

RSET

250 2.0.0 Reset state

QUIT

221 2.0.0 scarlatti.shakha.com closing connection

[Connection closed]

Share this post


Link to post
Share on other sites

BTW, the timing on this update to the filtering BLs is good, in that every false negative that has slipped by the filters into my inbox today (my wife's account also) has been from senders already listed on the Zen BL. Every one of them would have gone into Held if the SC servers were checking the Zen list. I'm looking forward to the update.

DT

Share this post


Link to post
Share on other sites

Because of this topic, I updated the filtering on my Exchange server from sbl-xbl to the Zen list yesterday. It has made a HUGE difference for the better in the amount of spam getting through to my user's inboxes.

Share this post


Link to post
Share on other sites
Because of this topic, I updated the filtering on my Exchange server from sbl-xbl to the Zen list yesterday. It has made a HUGE difference for the better in the amount of spam getting through to my user's inboxes.

I thought I had done likewise when cleaning up the RBL settings on the network I administer, but I just checked, and that upgrade was to the SBL-XBL, so I've just replaced that with ZEN and I'll see what happens.

DT

Edited by DavidT

Share this post


Link to post
Share on other sites
How did you check the email address? Unless you actually receive a response from him, he might be blackholing all incoming messages.

www.mailtester.com

Does a test similar to the one Farelf performed. Some mail hosts won't return any answer (such as Yahoo), but when others reply I have found it to be quite accurate.

Share this post


Link to post
Share on other sites
www.mailtester.com

Does a test similar to the one Farelf performed. Some mail hosts won't return any answer (such as Yahoo), but when others reply I have found it to be quite accurate.

I note a neat little bulk tester is available for download there too. The beauty of hexillion for single address tests is that additional options can be called for the single pass - such as test for catch-all, require MX records (both stipulated in my test above), etc.

On topic - great to see this topic is benefiting people WRT the Zen bl. Well done guys!

Share this post


Link to post
Share on other sites

The Zen blacklist is an available option now, if you want to try it out. We are going to wait a few days before making an announcement on the news page to make sure all the kinks are worked out. For now you can still subscribe to the other two Spamhaus blacklists, but if you enable Zen you should disable the others since they are included. We will remove the older ones and move everyone over to Zen prior to making a formal announcement.

Keep us updated on whether or not you are seeing good results. I ran a test on a batch of 1,000 spam messages. With SBL+XBL 174 messages made it to the Inbox, with Zen only 80 made it by.

-Trevor

Share this post


Link to post
Share on other sites

Actually, we have had a change of plans. We can't mirror "Zen" directly since it is really an aggregation of Spamhaus' three blacklists, so we are just going to add the PBL and you can choose whatever combination of the three you want (SBL+XBL+PBL = Zen). PBL is disabled for now until we get the local mirror up.

-Trevor

Share this post


Link to post
Share on other sites

OK, Trevor...close enough! :)

And....here's a happy sign found in a message in my Held mail this morning:

X-SpamCop-Disposition: Blocked pbl.spamhaus.org

There were a few more there that *almost* received that distinction, in that their SA scores were close to my threshhold, and the IPs were on the PBL.

Thanks,

DT

Share this post


Link to post
Share on other sites
There were a few more there that *almost* received that distinction, in that their SA scores were close to my threshhold, and the IPs were on the PBL.

Yep, PBL has been turned on and should be working.

-Trevor

Share this post


Link to post
Share on other sites
Yep, PBL has been turned on and should be working.

...except when it doesn't. :( See this TU:

http://www.spamcop.net/sc?id=z1499962664z2...57637d6517f914z

It's one that I just grabbed out of my wife's SC Email inbox. You'll see that the Barracuda that initially received the message got a positive from the PBL:

0.80 RCVD_IN_PBL RBL: Received via a relay in Spamhaus PBL

[84.190.93.96 listed in zen.spamhaus.org]

but SpamCop's "blade6.cesmail.net" server apparently didn't, or this would have been in my wife's Held mail folder. Several days ago, just after Trevor created the option, I selected the PBL for both of our accounts, and a few messages have been caught due to that, but not this one. Here's the lookup results on the IP at Spamhaus:

http://www.spamhaus.org/pbl/query/PBL043009

A lookup of the IP [84.190.93.96] at Robtex.com shows it's currently listed on:

dnsbl-2.uceprotect.net

dnsbl.sorbs.net

dul.dnsbl.sorbs.net

no-more-funn.moensted.dk

spamcop

The SCBL listing is about 5 hours old, but I think the message hit the SC email server just before that listing was activated. Regarding SORBS....does anyone remember when/why SORBS was removed from our BL choices? I have proof that it used to be there, but a search of the forums isn't turning up any historical discussion of what happened to that option. I found this topic in the Email forum:

IP listed in SORBS, but not filtered?

but it doesn't make any mention of removing SORBS from our BL list. I think that SORBS is still a fairly useful BL, isn't it?

DT

Share this post


Link to post
Share on other sites
...Regarding SORBS....does anyone remember when/why SORBS was removed from our BL choices? ...
Nope, don't recall that, hopefully someone might. It always was controversial, as a quick check of the history reminded me.
... I think that SORBS is still a fairly useful BL, isn't it?
I had the impression you used to be critical -
A quick little response from another SC email user -- I tried having the "dnsbl.sorbs.net" selected as one of my blacklists, but that one seems to cause too many false positives. I recommend simply "un-selecting" it in your Options.
Certainly there were fans and critics.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×