Jump to content

questions about the non-SC blacklists


Recommended Posts

(note: I've edited this message, and some of my others in this thread, now that I have a better understanding. In fact, even the Subject line of the thread probably out to be something like "questions about the non-SC blacklists." DT)

After seeing more and more spam getting through my settings (I really WISH we could have full Bayesian filtering!!!!), I went into my Options and added another blacklist, then Submitted the changes. I'm now using the following settings:

SpamAssassin Limit = 5

(yes, I realize that I can lower that to catch more spam)

SpamCop Blacklist

Composite Blocking List

SORBS DNSbl

I've had the SC BL and the SORBS DNSbl turned on all along -- I just added the Composite after analyzing some of the spam that gets through to my Inbox. However, when I used the Web reporting system on one of the spams that got through (well after the change I mentioned above), I noticed this line in the details:

217.95.235.156 listed in dnsbl.sorbs.net ( 127.0.0.10 )

I went to the SORBS site and the info on that IP is as follows:

217.95.235.156 found in Dynamic IP Space (Cable, DSL & Dial Ups)

Entry Created Sat Jun 19 01:19:30 2004 GMT

Last Updated Sat Jun 19 01:19:30 2004 GMT

So...if I've got the SORBS DNSbl checked in my Blacklists configuration for my SpamCop mailboxes (which I do), I wonder why the message made it through? It was NOT whitelisted, BTW. Here are the X-lines from the SC processing of the message:

X-spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on blade1

X-spam-Level: **

X-spam-Status: hits=2.1 tests=DATE_IN_PAST_06_12,HTML_60_70,HTML_FONT_BIG,

HTML_MESSAGE,J_CHICKENPOX_54,J_CHICKENPOX_74,LIMITED_TIME_ONLY,

MIME_HTML_ONLY version=2.63

X-SpamCop-Checked: 192.168.1.101 209.239.41.66 217.95.235.156

and here are the munged headers from the web reporting process:

Received: from telus.net (pD95FEB9C.dip0.t-ipconnect.de [217.95.235.156])

by host2.oneononeinternet.com (8.12.10/8.12.10) with SMTP id i64LXECW006019

for <x>; Sun, 4 Jul 2004 17:33:26 -0400

Message-Id: <2004___________________6019[at]host2.oneononeinternet.com>

From: "Ashley" <gatewlq[at]telus.net>

To: <x>

Subject:

Date: Sun, 4 Jul 2004 09:48:26 -0500

Mime-Version: 1.0

Content-Type: text/html; charset=us-ascii

I took a look at the "Held Mail" screen on which you can quickly see why each message has been routed to Held, (using the URL "http://mailsc.spamcop.net/reportheld?action=heldlog/") and all of them were either "Blocked bl.spamcop.net" or "Blocked SpamAssassin=" without a trace of any being blocked due to being on the SORBS DNSbl.

I'm going to run each of the spams that get past my settings through the web reporting interface, to see if messages from other IP's listed in SORBS and in the Composite BL get through, and I'll report them here in hopes that someone can explain the "leaking." And YES, I've read the draft of the "SpamCop Filters Leaky?" message and my situation doesn't seem to be described by what's there.

(update: 7/6/04 - I now understand that the "127.0.0.10" result from SORBS isn't very significant...and certainly not enough to block a message. I've now completed an anaysis of 401 messages in my Held Mail, and 5 of them were indeed put there due to my having opted for the SORBS DNSbl, so much of this thread is now moot...but not quite all.)

TIA,

David T.

Link to comment
Share on other sites

(I really WISH we could have Bayesian filtering!!!!)

I thought it was actually in place, for example the J_CHICKENPOX_74 I thought was an added in set of parameters for the Bayesian filter set ..???? Very few folks use 5 for the SpamAssissin level (based on the various "conversations here and in the newsgroups), though this doesn't seem to be a direct issue with your specific sample.

I've read the draft of the "SpamCop Filters Leaky?" message and my situation doesn't seem to be described by what's there

My recollection is that this item covered a lot of ground. That your situation isn't covered seems odd, especially if you're not going to jump in and add "why not" ... For instance, it would appear to me that your question might center on the lack of a Disposition Line, based on your focus on the SORBS listing. With a lack of background on the e-mail side of things, here's a thought that can get the controversy started .. just because an IP is "found in dynamic space" I'm not sure that this would return the same "status code" as an IP that was found to be an Open Relay .. so perhaps that's why the "not blocked" issues came up? If no one else has reported the spew from that IP yet, that would be the reason that the SpamCopDNSbl hasn't picked it up either. For example, one of the latest virus alerts (Bagle.x!proxy) has been defined by the fact that this one no longer goes madly spewing stuff out, it simply sets up the e-mail proxy server, send out notice that it's available, and just waits to be activated and used. Thus there may be thousands of infected machines out there that have yet to be activated.

For your future posts, simply providing the Tracking URL will have all the data needed to be seen available to see the spam, headers, scoring, etc. Much less cut/pasting on your end, much less screen space lost while "reading" here.

Link to comment
Share on other sites

(I really WISH we could have Bayesian filtering!!!!)

I thought it was actually in place, for example the J_CHICKENPOX_74 I thought was an added in set of parameters for the Bayesian filter set

I haven't followed the "Bayesian" issue lately....back a while ago, there was an aborted experimental attempt to fully implement Bayesian filtering, but there were some problems and I believe that it was abandonded. Perhaps some bits and pieces of Bayesian techniques have indeed been implemented since then, but from the amount of junk that gets through, I'm skeptical.

Very few folks use 5 for the SpamAssissin level (based on the various "conversations here and in the newsgroups)

I've experimented with various SA levels....my goal is NOT to have to go in and rummage around my Held Mail for false positives. There are almost 5,000 spams in there now (update 7/6/04 - I cleared out my Held Mail and my Trash), and the web system is extremely slow in dealing with more than a few items in Held Mail, in my experience. I've tweaked the display to show me 100 items at a time, in reverse order by arrival, with a preview of the first line, but with almost 50 pages to go through, that's a bit daunting. I was using SA=4 for a long time, but a few desireable items were getting trapped, so I decided to try adding to the blacklists instead, carefully.

For instance, it would appear to me that your question might center on the lack of a Disposition Line, based on your focus on the SORBS listing.

Yes, that's the mystery. Desipite my opting for the SORBS blacklist in my settings, a message got through that showed these lines in the web reporting system:

(update: I've cut out part of this message, in that it was related to my incomplete understanding of the 127.0.0.10 code from SORBS)

For your future posts, simply providing the Tracking URL will have all the data needed to be seen available to see the spam, headers, scoring, etc.  Much less cut/pasting on your end, much less screen space lost while "reading" here.

I don't save my Tracking URL's, so let's see...if I log into the web reporting system, click on "Past Reports" and then "View Recent Reports," I see a number associated with the report in question: 1095362364

Is that what you mean? That's not an URL, but it's the number associated with the report that I submitted.

(I'm now saving some of my Tracking URL's as I process items, but that's a pain in the rear....)

David T.

Link to comment
Share on other sites

Another "SORBS positive" spam just made it through the SC system and into one of my mailboxes. The long Tracking URL is:

http://www.spamcop.net/sc?id=z533513805z75...a79f10d7f813ebz

Again, the source IP is shown as being listed in SORBS:

62.139.24.89 listed in dnsbl.sorbs.net ( 127.0.0.10 )

but the message was allowed through. Should that be happening?

(update 7/6/04: yes, of course...I get this part now...)

DavidT

Link to comment
Share on other sites

OK, that last link you provided is just what "we" are looking for, the Tracking URL ... note the sorbs "result", ending with a ".10" .... Most BLs work with a simple set of reactions, no response if the IP isn't listed, a response if it is listed ... the pretty much agreed to 'standard' of a listed item is 127.0.0.2 ... but,, as seen here, some lists have various categories involved in their listings. As stated before, just that an IP is "listed" doesn't necessarily mean that there's an immediate "red alert" flag needed. For example, I know that my IP is in several lists, though not having changed in months, it's still considered as a dynamic IP, therefore should not be seen as any kind of server.

There's more that could be said, but ... I'm dealing right now with a 200+ pound white German Shepard that is not happy with some thunder mixed in with the kids down the street tossing out fireworks.

Link to comment
Share on other sites

Another spam got through, despite the IP source being "listed" in "cbl.abuseat.org" and my blacklist options include the CBL. Here's the Tracking URL:

http://www.spamcop.net/sc?id=z533647703z15...1b866ebbb59bf2z

Here's the line I'm talking about:

4.4.78.88 listed in cbl.abuseat.org ( 127.0.0.2 )

Here's the CBL data:

IP Address 4.4.78.88 was found in the CBL.

It was detected at 2004-07-04 22:00 GMT (+/- 30 minutes).

Perhaps the listing is too "fresh" for the CBL lookup done by SpamCop's system, and so that *might* explain how the spam got through, despite my having opted for CBL blocking in my SC email options....or maybe, the third-party blacklists aren't working properly?

(update 7/6/04: I'm now seeing some items in my Held Mail that got there thanks to the CBL, but the SC implementation of the CBL seems to be hours behind what's available from the blocklist source, according to my time calculations.)

David T.

Link to comment
Share on other sites

small problem .. looking at this last spam submittal (and a later look at the first Tracking URL offered), I don't see any obvious sign that the SpamCop E-Mail system handled this e-mail .... maybe this feeds back into my first response about your stating that the FAQ-in-progress didn't apply .. how is your e-mail being handled that you claim that SpamCop is missing any filtering modes? If you go back to the FAQ you looked at, you'll notice that none of the SpamCop "processing" lines are showing in this spam example ... I'm obviously missing something here.

Link to comment
Share on other sites

When pasting a message for analysis into the web reporting system, I sometimes take out the unnecessary lines (which is why you didn't see them), such as the "X-spam" lines (update 7/6/04: I'm no longer "helping" the parsing engine by removing any superfluous material from message headers...even though it's now resulting in my host servers getting sent in for relay testing, etc.)

...here they are:

X-spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on blade6

X-spam-Level: ****

X-spam-Status: hits=4.9 tests=BIZ_TLD,HTML_MESSAGE,J_BACKHAIR_12,

J_BACKHAIR_13,J_BACKHAIR_22,J_BACKHAIR_32,MIME_HTML_ONLY version=2.63

X-SpamCop-Checked: 192.168.1.101 66.84.24.227 4.4.78.88 4.4.78.88 206.122.60.47 4.4.78.88

I can see that this one would have been caught if I would lower my SA number to 4, but according to the details from the web reporting system, the source IP would have already been in the CBL system for about four hours at the point that the message arrived at the SpamCop email servers. The IP was "detected at 2004-07-04 22:00 GMT" by the CBL system, and the email arrived at the servers at 5 Jul 2004 02:05:24 -0000. I'm not sure if SpamCop queries the CBL directly or uses a cache or a mirror, but it shouldn't be that far behind, according to what I read at the CBL site. My contention is that this spam should have gone into my Held Mail due to my selection of the CBL in my blacklist options.

Link to comment
Share on other sites

There are almost 5,000 spams in there now

How many days did it take to build up to 5,000 spams.

If you are able to use IMAP I would recommed using it to scan and delete mail from the held mail file. Bulk deletion is much faster.

Have you checked to make sure that you are not downloading mail from multiple sources? It looks like the mail you reverenced above was downloaded from hypercreations.com and not from Spamcop.net

The blacklists only work on mail that is processed through your spamcop account.

How do you send your mail to your spamcop email account?

Link to comment
Share on other sites

When pasting a message for analysis into the web reporting system, I sometimes take out the unnecessary lines (which is why you didn't see them), such as the "X-spam" lines..

If I understand your statement correctly you are refering to editing a message that your are reporting as spam.

Note: there are NO unnecessary lines when reporting spam.

The spam message should never be edited (the only exception being that of munging personal data, which if done must also include the additional disclaimer that you have munged the data.

Link to comment
Share on other sites

How many days did it take to build up to 5,000 spams.

Over a month...I don't go in and clean up my Held Mail any more (update - I've now cleaned out the backlog), because the web system is WAY too slow and life is too short. From my readings of FAQs and messages, I'm led to believe that it's OK to just leave a ton of message in Held or Trash, because it's supposed to expire eventually....right? (update: inquiring minds would still like the answer to this one....I read someone here on the forum that even Held Mail should expire after about 14 days, but it's not doing so.)

If you are able to use IMAP I would recommed using it to scan and delete mail from the held mail file.  Bulk deletion is much faster.

I use Pegasus, which has IMAP support, but I've not tried it with my SC box (update: I got off my lazy a** and accessed my SC mail using the Pegasus IMAP support). I POP the stuff in my Inbox and occasionally use the webmail system to take a look at the Held...I'd rather not have to look at the Held at all, which is why I'm trying to get the blocking to work better.

Have you checked to make sure that you are not downloading mail from multiple sources?  It looks like the mail you reverenced above was downloaded from hypercreations.com and not from Spamcop.net

I grabbed it from SpamCop....I chopped a few Received lines off the top when pasting it in for reporting. I got burned a few times by the reporting system when it falsely reported my HyperCreations.com IP and put it on the SCBL, so I only put what the system really needs to see in there.

The blacklists only work on mail that is processed through your spamcop account. How do you send your mail to your spamcop email account?

It gets automatically forwarded there from various aliases on various domains, including HyperCreations.com

David T.

Link to comment
Share on other sites

If I understand your statement correctly you are refering to editing a message that your are reporting as spam. Note: there are NO unnecessary lines when reporting spam.

If I'm breaking a rule then I'm sorry. I got screwed by the system when the system managed to falsely identify my own domain as the source. This happened when I used the Report spam function of the webmail system, in which you don't get to see where reports are going. So, I hardly ever report anything anymore. Yes, I know about the "mole" option.

In order to properly parse the most recent spam mentioned above, why would the system need this:

Received: from unknown (192.168.1.101)

by blade6.cesmail.net with QMQP; 5 Jul 2004 02:05:24 -0000

That's internal to the SC system. Ditto for this:

Delivered-To: (deleted)

Received: (qmail 18889 invoked from network); 5 Jul 2004 02:05:24 -0000

Closer to the true source is found this line:

Received: from s227.n24.vds2000.com (HELO hypercreations.com) (66.84.24.227)

by mailgate.cesmail.net with SMTP; 5 Jul 2004 02:05:24 -0000

That's the handoff from my domain to SC, and that's the line that the system has mistaken in the past and put my own domain in the SCBL, when it had problems parsing the true source of the spam. I don't want to get my own system in the SCBL, because then I'm screwed.

Link to comment
Share on other sites

When pasting a message for analysis into the web reporting system, I sometimes take out the unnecessary lines (which is why you didn't see them), such as the "X-spam" lines...here they are:

Attitude check in process .. you've asked for help, got around to pointing to your examples, but only now state that even those samples are much modified. Think I'm done trying here.

From my readings of FAQs and messages, I'm led to believe that it's OK to just leave a ton of message in Held or Trash

Seems to be a fairly strange take on the situation, as almost all posts I can recall deal with the problems caused by the build-up of too many e-mails collected.

I got screwed by the system when pasting entire messages in, when the system managed to falsely identify my own domain as the source.

And the reason for this screwing was ...???? Implies that there is a problem in that ISP's configuration, perhaps the Mail-Host thing might have handled it, maybe your ISP needs a heads-up, but again, it's a bit late in the game to get around to defining all your editing of items submitted for help/analysis ...

218.72.218.194 listed in dnsbl.sorbs.net ( 127.0.0.10 )

You overlooked a post I made that talked a bit about result codes?

Maybe someone else can help, I think I'm done wirth this one ...

Link to comment
Share on other sites

It seems to me also that you have not set up mailhosts yet.

Although it is not required to be done at this time, it will probably soon be a requirement. The whole purpose of mailhosts is to avoid reporting yourself. Spamcop is made aware of who your mail servers are in the mailhost registration process and then those servers are removed from being report on any mail submitted by you.

Also remember timing is an issue when using the parser.

Spamcop blacklists are applied at the time the mail is received on the spamcop server.

The parser shows listed items as of the time the parse was run.

The result is a spam may not have been listed at the time it was received but is listed at the time the parse is run.

Link to comment
Share on other sites

So, I hardly ever report anything anymore. Yes, I know about the "mole" option.

Wasn't mole reporting declared dead a while back? Mind you, the relevant FAQ seems to be back to its old self... updated "2004-Jul-02 1:51pm."

If good old Moley is alive again, could we all hold a party to celebrate? If OTOH the FAQ is wrong (if it's somehow reverted to a primordial state, say), this may need a tweak...

Cheers, Nick

Link to comment
Share on other sites

.. you've asked for help, got around to pointing to your examples, but only now state that even those samples are much modified.  Think I'm done trying here.

Wait, please don't give up just yet. The modifications didn't affect the parsing of the web reporting system's "bottom line." The lines I deleted would all have eventually been ignored by the parser, in that they had nothing to do with the source of the spam. When you asked questions, I answered as best I could.

Seems to be a fairly strange take on the situation, as almost all posts I can recall deal with the problems caused by the build-up of too many e-mails collected.

I'll go through the old posts, if necessary, but I'm pretty sure that people have said that clearing out the Trash or Held Mail is optional.

And the reason for this screwing was ...????

The parser had problems finding the true source, and so it falsely reported my own host. This happened with quick reporting, and I verified that it was not my own "ISP's configuration." I don't know if the Mail-Host thing would help -- I receive mail at a lot of domains, and the instructions aren't all that clear, so I've not set that up yet.

You overlooked a post I made that talked a bit about result codes?

I saw the post, but I haven't yet been convinced that an "is listed" result in the parser report doesn't mean that the same blacklist would catch the message.

Maybe someone else can help, I think I'm done wirth this one ...

Guess I'll have to start over again, perhaps in the newsgroups with deputies, because I really would like to find out why the spams are getting through, even though they are "listed" with the BL's.

dt

Link to comment
Share on other sites

Wasn't mole reporting declared dead a while back?

Note sent off to Deputies/RW asking about it. A while back, Richard was holding off on some updates awaiting a hard drive replacement, and I'll agree, my first thought is that this currnt FAQ looks like something that came from an archive ...

Link to comment
Share on other sites

Wait, please don't give up just yet. The modifications didn't affect the parsing of the web reporting system's "bottom line."

For lack of a better word, I'll use "trust" ... You say "this is what I got" ... "this is what I submitted" ... but leave out that what we/I was supposed to be analyzing was missing stuff that you felt wasn't needed .. which kind of flies right in the face of asking/analyzing/answering a "why" question.

I'm pretty sure that people have said that clearing out the Trash or Held Mail is optional.

Yet I know that I forwarded requests to JT for direct intervention for a number of folks that had let the collection get to big ... I have a Pinned item from JT talking about even going to a setting to "not" send deleted e-mails to the Trash folder to help keep it from building up ...

I saw the post, but I haven't yet been convinced that an "is listed" result in the parser report doesn't mean that the same blacklist would catch the message.

I went to SORBS, the ".10" says that it's within a block of dial-up assignments, just as I conjectured. No, I can't say specifically what the SpamCop filter set does with a .10 result, other than to suggest that it's not treated as a "flag" .. possibly just more of an indicator that the e-mail may be suspect, as compared to a known spam spew source.

The parser had problems finding the true source, and so it falsely reported my own host. This happened with quick reporting

It would have been much better to work out that issue. There are many folks of the view that the quick-reporting option ought to withdrawn, much of that driven by situations just as this.

Guess I'll have to start over again, perhaps in the newsgroups with deputies

Though not often, and definitely not everywhere, (at least a couple) of the Deputies make their way over here, Don has posted in a couple of spots. Yes, you'll have more "old-timers" over in the newsgroups, I do both places ... just a heads-up .... if you submit a spam for analysis over there, do not do what you did here .... provide a full and complete set of headers for the parse and the example .. this is supposed to be a calm, nice place ... the newsgroups don't have that "request" in place ...

Link to comment
Share on other sites

I went to SORBS, the ".10" says that it's within a block of dial-up assignments, just as I conjectured.  No, I can't say specifically what the SpamCop filter set does with a .10 result

But that's exactly what needs to be explained, in order to answer my question, so maybe there's someone in the newsgroups who has that information.

I posted it in "spamcop.help" because I need "help" with "SpamCop," but I wondered if I should post it to "spamcop.mail" instead? On this topic, there seems to be a mis-match between the list of newsgroups mentioned on SpamCop's website, where the following four groups are mentioned:

SpamCop

Geek talk

Social room

spam lab

No mention is made of either "spamcop.help" or "spamcop.mail" (or even "spamcop.routing" for that matter), all of which are valid groups. I just checked the current items in the "spamcop" group...no FAQ there.

David T

Link to comment
Share on other sites

No mention is made of either "spamcop.help" or "spamcop.mail" (or even "spamcop.routing" for that matter),

It was JT's contention that "this" is the place for support of various items, starting with the e-mail side of the house. Both the e-mail accounts and newsgroups are running on his servers. Long story, much contested, dating back to the genisys of these Forums back around the end of last year.

Link to comment
Share on other sites

(I just posted the following to the NNTP group, "spamcop.help" and it's appropriate here)

a brief followup....I ran my own cable broadband IP through the SC web system and came up with this:

listed in dnsbl.sorbs.net ( 127.0.0.10 )

So it would be a bad thing if that particular type of SORBS hit was enough to trigger blocking on the SpamCop server. However, I'd still like to know the details of what kind of SORBS listing would actually trigger blocking, given that I've got SORBS DNSbl on my blocking list configuration for my SC mailboxes. I did a little Googling, and found various hits for the following:

"listed in dnsbl.sorbs.net ( 127.0.0.6"

"listed in dnsbl.sorbs.net ( 127.0.0.7"

"listed in dnsbl.sorbs.net ( 127.0.0.10"

All those hits are probably to SpamCop parser results. Interesting enough, one of them was to the SC web forums, on an almost identical topic to my own:

http://forum.spamcop.net/forums/index.php?showtopic=872

I read through that thread, and the person's question was never fully answered.

Link to comment
Share on other sites

I see the same issue ... the SORBS result of ".10" .. the same remarks that this is not a "red flag alert" code, the same possibility of timing offered, never mind the confusion factor of the original poster much reduced with all the other stuff be handled in that dialog.

Link to comment
Share on other sites

I see the same issue ... the SORBS result of ".10" .. the same remarks that this is not a "red flag alert" code, the same possibility of timing offered, never mind the confusion factor of the original poster much reduced with all the other stuff be handled in that dialog.

Yes, a lot of overlap with the thread I started, and yet I don't see an authoritative explanation of how SC interacts with/uses the SORBS DNSbl. I think we need a system admin to weigh in on this.

Link to comment
Share on other sites

Wasn't mole reporting declared dead a while back?

Note sent off to Deputies/RW asking about it. A while back, Richard was holding off on some updates awaiting a hard drive replacement, and I'll agree, my first thought is that this currnt FAQ looks like something that came from an archive ...

Richard is not happy <g> .. agrees that it looks like the FAQ was re-built off of an old archive .. He appreciates that "we've" given him something to do over the next few days <g>

Link to comment
Share on other sites

Yes, a lot of overlap with the thread I started, and yet I don't see an authoritative explanation of how SC interacts with/uses the SORBS DNSbl. I think we need a system admin to weigh in on this.

lease go back and catch my post Jul 5 2004, 11:30 AM ... one person, JT ... and just a side-note, this is a long week-end. As you've seen, you're asking about a filtered e-mail situation and there's not a lot of folks over in the newsgroups that have any more insight to answer answer the question either ... Several of your issues have been touched, even answered .. the last specific one has been answered .. the ".10" result from SORBS is not a spam flag ... beyond that, not sure what else to carry on here with .. especially after noting that I'm already beyond "the half-dozen posts so I've already exceeded my attention span" point (not sure whether to put any kind of emoticon after that or not)

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...