IP listed in SORBS, but not filtered?

[silentlarry]

After several spam-free years with an SC account and no filters needed, my aunt is kinda freaking out to be getting hit with the new pump & dump crop. I've turned on all the filters, tightened up spamassassin as much as I'm willing (now 3).

Today one came in from 83.35.247.12 just missign the SA setting at the time.

http://www.spamcop.net/sc?id=z1139707441zb...dc9f3b23e1c27fz

spam was 3 hours old at the time I checked it, CBL said the ip had been listed for 4 hours but that's just refresh/caching issues making it a near-miss, I assume.

However, SORBS says it's been listed since Oct 28

http://www.sorbs.net/lookup.shtml?83.35.247.12

... and just to anticpate the FAQs

* yes, I looked at the FAQ

* sorbs filter was on

* "tag only" is not checked

* no, it did not trigger a whitelisting

Anyway, nothing to get fussy over but I'm wondering what's going on with the SORBS bl checking.

Thoughts?

[Telarin]

This had a SpamAssassin score of 3.6:

X-spam-Status: hits=3.6 tests=HELO_DYNAMIC_SPLIT_IP,HTML_MESSAGE, SARE_GIF_ATTACH version=3.1.1

What is your filtering threshhold set at?

[silentlarry]

This had a SpamAssassin score of 3.6:

What is your filtering threshhold set at?

At the time it was 4 or 5, so it's no mystery why SA did not block it. Since been changed to 3.

Over the course of the last few weeks the spammys have probably been tuning for minimum spamassain smoke. Initally they were averaging around a 15 score, which was easy pickings with threshold at 5 and no blocklists. But it edged lower and lower, currently averaging around 4, the lowest was 1.something. (Again, this is all the same pump & dump gif spam over and over, with a different stock each week.)

So I'm hoping the blocklists will take up the slack with the SA scoring lower and lower. It was a little frustrating to have my aunt forward me one from an IP that appears to have been listed in sorbs but did not get blocked.

Let me try a more specific question... Anybody else seeing hits on SORBS BL blocking lately?

If so I'll assume this was an unfortunate glitch, and shut up.

Thanks

[StevenUnderwood]

Let me try a more specific question... Anybody else seeing hits on SORBS BL blocking lately?

If so I'll assume this was an unfortunate glitch, and shut up.

In my small sample (17 messages since Sunday morning) I have not seen it. Here are the dispositions for those 17.

X-SpamCop-Disposition: Blocked SpamAssassin=33

X-SpamCop-Disposition: Blocked cbl.abuseat.org

X-SpamCop-Disposition: Blocked SpamAssassin=8

X-SpamCop-Disposition: Blocked SpamAssassin=6

X-SpamCop-Disposition: Blocked cbl.abuseat.org

X-SpamCop-Disposition: Blocked SpamAssassin=11

X-SpamCop-Disposition: Blocked SpamAssassin=10

X-SpamCop-Disposition: Blocked SpamAssassin=6

X-SpamCop-Disposition: Blocked cbl.abuseat.org

X-SpamCop-Disposition: Blocked SpamAssassin=5

X-SpamCop-Disposition: Blocked SpamAssassin=18

X-SpamCop-Disposition: Blocked SpamAssassin=5

X-SpamCop-Disposition: Blocked SpamAssassin=13

X-SpamCop-Disposition: Blocked SpamAssassin=25

X-SpamCop-Disposition: Blocked SpamAssassin=15

X-SpamCop-Disposition: Blocked SpamAssassin=18

X-SpamCop-Disposition: Blocked SpamAssassin=14

[petzl]

Not an answer for why an IP listed by SORBS since 28th of last month is not blocking near a month on?

[silentlarry]

Another example.

Experimenting on my own acount today (vs my aunt's... I know what's good for me!) , I've turned off all filters except dnsbl.sorbs.net

I just got spam from 87.7.164.70 which did not get tagged as SpamCop-Disposition: blocked

http://www.spamcop.net/sc?id=z1141864742z8...4e606c9615bfa7z

SORBS shows;

Netblock:	87.0.0.0/12 (87.0.0.0-87.15.255.255)
Record Created:	Sat Jan 28 22:32:08 2006 GMT
Record Updated:	Sat Jan 28 22:32:08 2006 GMT
Additional Information:	Dynamic/Generic IP/rDNS address, use your ISPs mail server or get rDNS set to indicate static assignment.
Currently active and flagged to be published in DNS

I'm no guru but at this point I'm going to conclude that spamcop mail's sorbs filtering is broken, reason unknown. What next?

My thanks to those helping with this...

[StevenUnderwood]

I'm no guru but at this point I'm going to conclude that spamcop mail's sorbs filtering is broken, reason unknown. What next?

Well, the last one in my inbox at home that would have been caught by sorbs since you posted the problem:

Return-Path: <x[at]hotmail.com>

Delivered-To: spamcop-net-y[at]spamcop.net

Received: (qmail 10010 invoked from network); 20 Nov 2006 03:05:02 -0000

X-spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on blade3.cesmail.net

X-spam-Level: **

X-spam-Status: hits=2.6 tests=HTML_MESSAGE,J_CHICKENPOX_43,MISSING_SUBJECT,

MSGID_FROM_MTA_HEADER,SARE_UNSUB38D version=3.1.1

Received: from unknown (192.168.1.103)

by blade3.cesmail.net with QMQP; 20 Nov 2006 03:05:02 -0000

Received: from bay0-omc3-s26.bay0.hotmail.com (65.54.246.226)

by mx53.cesmail.net with SMTP; 20 Nov 2006 03:05:01 -0000

Received: from hotmail.com ([65.55.132.29]) by bay0-omc3-s26.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.1830);

Sun, 19 Nov 2006 19:05:00 -0800

Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;

Sun, 19 Nov 2006 19:05:00 -0800

Message-ID: <BAY127-DAV1949A813E426E3345AFA1CB3ED0[at]phx.gbl>

Received: from 141.154.220.93 by BAY127-DAV19.phx.gbl with DAV;

Mon, 20 Nov 2006 03:04:58 +0000

X-Originating-IP: [141.154.220.93]

X-Originating-Email: [x[at]hotmail.com]

X-Sender: x[at]hotmail.com

From: "X" <x[at]hotmail.com>

To: "X" <x[at]hotmail.com>

Subject:

Date: Sun, 19 Nov 2006 22:04:52 -0500

Message-ID: <001f01c70c50$a5ef2a20$6501a8c0[at]MOM>

MIME-Version: 1.0

Content-Type: multipart/related;

boundary="----=_NextPart_000_0020_01C70C26.BD192220"

X-Mailer: Microsoft Office Outlook 11

X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962

thread-index: AccMUKXBRB2ECl7lQRC8XRN9iAREkQ==

X-OriginalArrivalTime: 20 Nov 2006 03:05:00.0360 (UTC) FILETIME=[AAAB0480:01C70C50]

Return-Path: x[at]hotmail.com

X-SpamCop-Checked: 192.168.1.103 65.54.246.226

X-SpamCop-Disposition: Blocked dnsbl.sorbs.net

X-SpamCop-Whitelisted: x[at]hotmail.com

I have these others for this month as well.

11/19, blade3

11/17, blade3

11/16, filter7

11/15, blade3

11/15, blade3

11/15, blade4

11/13, blade6

11/11, blade2

11/10, blade4

11/8, filter7

11/1 blade1

Edit to include server names handling the messages.

[silentlarry]

Well, the last one in my inbox at home that would have been caught by sorbs since you posted the problem:

...

X-SpamCop-Disposition: Blocked dnsbl.sorbs.net

I have these others for this month as well. (11/17, 11/16, 11/15 x3, 11/13, 11/11, 11/10, 11/8, 11/1)

Ok well something works then (thank you for checking)... is it just spotty?

Am I checking the wrong thing at sorbs?

Global warming?

[Wazoo]

Another discussion about SpamAssassin scoring involved the fact that there are multiple servers involved .... is it possible that all of your 'failed to be caught' e-mails was handled by the same server?

[silentlarry]

EDIT by poster: I misunderstood this question... this post should probably be ignored!

Another discussion about SpamAssassin scoring involved the fact that there are multiple servers involved .... is it possible that all of your 'failed to be caught' e-mails was handled by the same server?

Just to clarify: The object of my concern are some (apparent) failures of SC mail to block based on dnsbl.sorbs.net. ...although if I were having better luck with spamassain the issue would not have come up. (you may be not be confused about that, just trying to keep it clear 'cause this has wandered around a bit)

I don't see this as a SpamAssassin issue. I am aware that SpamAssassin scoring is a squishy thing, results will vary as spam changes and SpamAssassin rules are changed in response over time. My only 'failed to be caught' issue is with sorbs checking.

I'd just as soon leave spamassain out of it but, what they hey in case anyone cares...

The inital spam which started this scored low-ish 3.6 sez:

X-spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on blade1

Some more recent from the same pumper dumpers going back to the 18th:

X-spam-Checker-Version:   	SpamAssassin 3.1.1 (2006-03-10) on blade2.cesmail.net
X-spam-Level:	  ****
X-spam-Status:	  hits=4.4 tests=EXTRA_MPART_TYPE,HTML_MESSAGE,SARE_GIF_ATTACH, TVD_FW_GRAPHIC_ID1 version=3.1.1

X-spam-Checker-Version:   	SpamAssassin 3.1.4 (2006-07-26) on filter8
X-spam-Level:	  ****
X-spam-Status:	  hits=4.5 tests=EXTRA_MPART_TYPE,HTML_MESSAGE,SARE_GIF_ATTACH, TVD_FW_GRAPHIC_ID1,TW_CS,TW_SJ version=3.1.4

X-spam-Checker-Version:   	SpamAssassin 3.1.1 (2006-03-10) on blade4
X-spam-Level:	  ****
X-spam-Status:	  hits=4.4 tests=EXTRA_MPART_TYPE,HTML_MESSAGE,SARE_GIF_ATTACH, TVD_FW_GRAPHIC_ID1,TW_KS version=3.1.1

X-spam-Checker-Version:   	SpamAssassin 3.1.1 (2006-03-10) on blade1
X-spam-Level:	  ***********
X-spam-Status:	  hits=11.2 tests=EXTRA_MPART_TYPE,HTML_IMAGE_ONLY_20, HTML_MESSAGE,MY_CID_AND_ARIAL2,MY_CID_AND_CLOSING,MY_CID_AND_STYLE, MY_CID_ARIAL2_CLOSING,MY_CID_ARIAL_STYLE,SARE_GIF_ATTACH, SARE_GIF_STOX,TVD_FW_GRAPHIC_ID1 version=3.1.1

X-spam-Checker-Version:   	SpamAssassin 3.1.1 (2006-03-10) on blade1
X-spam-Level:	  ******
X-spam-Status:	  hits=6.4 tests=HELO_DYNAMIC_HEXIP,HELO_DYNAMIC_IPADDR2, HTML_MESSAGE,SARE_GIF_ATTACH,SUBJ_ALL_CAPS version=3.1.1

My thanks again to those contributing brainpower.

[Wazoo]

You seemed to have missed the point .... I referenced the SpamAssassin Topic/Discussion because it highlighted that there are several servers involved in the handling of the incoming e-mail. Each server has some configuration files and settings. The end-goal of the suggested question ... is one of the servers needing a touch because the sorbs look-up isn't getting done? Therefore the question is again ... is it possible that all of your 'failed to get caught' e-mails is being handled by the same server?

[StevenUnderwood]

Therefore the question is again ... is it possible that all of your 'failed to get caught' e-mails is being handled by the same server?

I have modified my earlier post to include the servers seen this month.

[Wazoo]

I have modified my earlier post to include the servers seen this month.

So there's no confusion, that post does indicate a sorbs check done on/by 'blade3' ... but this e-mail was dropped into the InBox due to the Whitelisting rule that was also applied.

Received: from ..... by blade3.cesmail.net

X-SpamCop-Checked: 192.168.1.103 65.54.246.226

X-SpamCop-Disposition: Blocked dnsbl.sorbs.net

X-SpamCop-Whitelisted: x[at]hotmail.com

Most of the servers addressed do match, noting that 'filter8' is only mentioned in silentlarry's post.

[silentlarry]

You seemed to have missed the point ....

Yes, I did!

So far from my two examples of missed sorbs lookups:

X-spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on blade1

X-spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on blade1

...can I assume the server seen in the above header is the same one that checks the IP blocklists?

It's also the last server in the receive chain. If that's not right, could somebody who knows what they're doing please look at the the two tracking urls I've previously posted? Thanks.

Most of the servers addressed do match, noting that 'filter8' is only mentioned in silentlarry's post.

I think you can throw that post out, those were mostly blocked by spamassain and would not have been checked against sorbs. I quoted them as I'd mistakenly thought the conversation had taken a side turn to SA.

[Wazoo]

Yes, I did!

<g>

So far from my two examples of missed sorbs lookups:

X-spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on blade1

...can I assume the server seen in the above header is the same one that checks the IP blocklists

No .. still kind of missing the point ... You listed the invocation of the SpamAssassin check on 'the 'blade1' server ...

What you are 'complaining' about would be the lack of the line;

X-SpamCop-Disposition: Blocked dnsbl.sorbs.net

or, as in Steven's example, the existence of a line like;

X-SpamCop-Whitelisted: <someone's address or a Domain>

on an e-mail that you state should have been caight by a 'current/old' sorbs listing, which indicate that the server handling that e-mail may not be doing a sorbs look-up ...

It's also the last server in the receive chain. If that's not right, could somebody who knows what they're doing please look at the the two tracking urls I've previously posted? Thanks.

I think you can throw that post out, those were mostly blocked by spamassain and would not have been checked against sorbs. I quoted them as I'd mistakenly thought the conversation had taken a side turn to SA

http://www.spamcop.net/sc?id=z1141864742z8...4e606c9615bfa7z - used 'blade1' ...

http://www.spamcop.net/sc?id=z1139707441zb...dc9f3b23e1c27fz - used 'blade1' ...

(which I note that you did already identify)

off doing some other research ... back in a few ..

sorbs database check;

87.7.164.70

Netblock: 87.0.0.0/12 (87.0.0.0-87.15.255.255)

Record Created: Sat Jan 28 22:32:08 2006 GMT

Record Updated: Sat Jan 28 22:32:08 2006 GMT

83.35.247.12

Netblock: 83.35.0.0/16 (83.35.0.0-83.35.255.255)

Record Created: Thu Oct 28 07:52:28 2004 GMT

Record Updated: Thu Oct 28 07:52:28 2004 GMT

[silentlarry]

Looking thru today's non-spam mail, the following were tagged "Blocked dnsbl.sorbs.net"

filter8

blade1

blade3.cesmail.net

Ok so some are getting tagged for sorbs, even on blade1 which is in common with my "missed" examples.

I dunno what gives. To grasp at some straws...

How soon after changing the blacklist setting does it actually take effect? Is there some delay involved?

For now I'm leaving sorbs as the only list selected to see if it this is consistent. One thing I haven't done is go thru my non-spam that did not have a sorbs hit to see if any of those are listed, but that will take me a while.

Thanks folks...

[Wazoo]

From: "WazoO"

To: "SpamCop Support - JT"

Subject: blade1 - sorbs check

Date: Mon, 20 Nov 2006 20:58:11 -0600

Based on dialog in http://forum.spamcop.net/forums/index.php?showtopic=7518

there is a possible question as to whether blade1 is doing a

sorbs.net BL look-up.

Edit: Dang, looks like I should have waited .... based on the post made while was doing up the e-mail ...

From: "WazoO"

To: "SpamCop Support - JT"

Subject: Fw: blade1 - sorbs check

Date: Mon, 20 Nov 2006 21:08:57 -0600

Naturally, as soon as I sent the previous, the story changed a bit ...

Going to have to ask you for a bit of a direct answer.

One question might be; the sorbs data update cycle, assuming you

are using a cached version.

The first 'example' points back to a sorbs listing from last October,

for instance ...

> Based on dialog in

http://forum.spamcop.net/forums/index.php?showtopic=7518

> there is a possible question as to whether blade1 is doing a

> sorbs.net BL look-up.

[silentlarry]

No .. still kind of missing the point ... You listed the invocation of the SpamAssassin check on 'the 'blade1' server ...

What you are 'complaining' about would be the lack of the line;

X-SpamCop-Disposition: Blocked dnsbl.sorbs.net

Yes, that is essentially the gripe. The line I listed was picked out because I assumed that the server that is used for the spamassain check would be the same one doing the IP blocklist checks. (if that's not correct, please fill me in where I should look) The "missed" examples do indeed lack "Blocked dnsbl.sorbs.net"... that was the whole point.

or, as in Steven's example, the existence of a line like;

X-SpamCop-Whitelisted: <someone's address or a Domain>

on an e-mail that you state should have been caight by a 'current/old' sorbs listing, which indicate that the server handling that e-mail may not be doing a sorbs look-up ...

Not sure if I'm following you but... I have seen many many examples of "SpamCop-Whitelisted" emails which also indicate a disposition of "blocked"... I have seen several today. I always assumed this means the spam checks (first spamassain, then the blocklists, stopping with the first positive) are always performed even though it may be a whitelisted address. (Whitelisting will always pass it thru to the mailbox regardless of blocked dispostion.)

Is this not the case?

At any rate, of my two measly examples of spam that were not tagged even though listed in sorbs, were not whitelisted. (hope I understood what you were getting at!)

Thanks, and sorry about the moving target.

[silentlarry]

FYI another spam from an IP listed in SORBS was not flaged as blocked.

http://www.spamcop.net/sc?id=z1142626807z7...c78d56b8527ea3z

spam ip=222.232.153.216

server was blade5

SORBS sez:

Netblock:	222.232.153.0/24 (222.232.153.0-222.232.153.255)
Record Created:	Tue Jul 11 15:42:34 2006 GMT
Record Updated:	Tue Jul 11 15:42:34 2006 GMT
Additional Information:	[Hanaro Supplied list] Dynamic/Generic IP/rDNS address, use your ISPs mail server or get rDNS set to indicate static assignment.
Currently active and flagged to be published in DNS

Again:

spamassasin blocking is turned off for the purpose of testing

SORBS is the only blacklist selected

was not flaged as whitelisted

[StevenUnderwood]

FYI another spam from an IP listed in SORBS was not flaged as blocked.

http://www.spamcop.net/sc?id=z1142626807z7...c78d56b8527ea3z

spam ip=222.232.153.216

server was blade5

SORBS sez:

Netblock:	222.232.153.0/24 (222.232.153.0-222.232.153.255)
Record Created:	Tue Jul 11 15:42:34 2006 GMT
Record Updated:	Tue Jul 11 15:42:34 2006 GMT
Additional Information:	[Hanaro Supplied list] Dynamic/Generic IP/rDNS address, use your ISPs mail server or get rDNS set to indicate static assignment.
Currently active and flagged to be published in DNS

I suppose it is possible that Jeff is ignoring the dul list (response 127.0.0.10) or is using one of the other lists from sorbs that does not contain the dul code as all of the samples you have provided thus far are in that list exclusively, I believe. That could make sense as spamcop scans all IP addresses within the email and theoretically, most every end user machine should be in the dul list so if ISP's are properly noting where they are getting the mail from, it would be listed within spamcop and make that list almost worthless. If so, the selection should indicate exactly which list is being queried.

This is all guesswork right now and would need confirmation from JT. I will test my theory against the mail I have at home that was tagged from sorbs.

[silentlarry]

Ok, this is making way too much sense. I just sent myself an email from home via Mail App (to spamcops SMTP server), and it was not tagged as blocked even though it (my home IP) comes up in sorbs as dynamic IP (at least that's what http://www.moensted.dk/spam/ says... I think I went over some limit for quering the sorbs site, it's mad at me right now)

I think probably you've nailed it.

I suppose it is possible that Jeff is ignoring the dul list (response 127.0.0.10)

If that's the case, I think another "Messages not Filtered" FAQ item is in order.

or is using one of the other lists from sorbs that does not contain the dul code

Wheras if this turns out to be the case, then the blacklist selection menu is dreadfully misnamed as it points right to the aggregate list. Probably another item for the FAQ rather than wait for the menu to changed, eh?

... most every end user machine should be in the dul list so if ISP's are properly noting where they are getting the mail from, it would be listed within spamcop and make that list almost worthless.

Make me wonder what anyone would a list inclusive of DUL for... unless one can parse the headers so as to determine the difference between the end user's IP and mail servers. Which is something I assumed spamcop did, but running my mail thru the parser I see that it fingers my IP. And thinking about it I can see why that's a good thing. Anyway this is wandering way above my level of knowlege so I think I'll just shut up here.

>>If so, the selection should indicate exactly which list is being queried.

I think we are talking the same thing here. Anyhoo when the dust settles, somekinda FAQ addtion will probably be in order? Looks like this was a big fuss about nuttin'

Thanks Steve & everybody

[Telarin]

Make me wonder what anyone would a list inclusive of DUL for... unless one can parse the headers so as to determine the difference between the end user's IP and mail servers. Which is something I assumed spamcop did, but running my mail thru the parser I see that it fingers my IP. And thinking about it I can see why that's a good thing. Anyway this is wandering way above my level of knowlege so I think I'll just shut up here.

Using a DUL is fine for a mailserver that is filtering on connecting IP only, as you should never have a machine from a dynamic range sending mail direct to MX. There is simply no way for it to maintain the proper PTR records required.

[Wazoo]

Date: Tue, 21 Nov 2006 23:23:06 -0500

From: SpamCop Support

To: WazoO

Subject: Re: Fw: blade1 - sorbs check

References: <005b01c70d1a$6313c7e0$6401a8c0[at]msi6378>

In-Reply-To: <005b01c70d1a$6313c7e0$6401a8c0[at]msi6378>

We don't block based on the SORBS "dynamic" list. Because we check every

mail header, it's entirely reasonable for a dynamic host to appear in

the headers. I just checked and our copy of SORBS is less than an hour old.

Jeff

WazoO wrote:

> Naturally, as soon as I sent the previous, the story changed a bit ...

> Going to have to ask you for a bit of a direct answer.

>

> One question might be; the sorbs data update cycle, assuming you

> are using a cached version.

>

> The first 'example' points back to a sorbs listing from last October,

> for instance ...

>

>> Based on dialog in

> http://forum.spamcop.net/forums/index.php?showtopic=7518

>> there is a possible question as to whether blade1 is doing a

>> sorbs.net BL look-up.

[silentlarry]

Thanks...