Jump to content

Parsing source IP vs. full spam


DavidT

Recommended Posts

This is something that has puzzled me for a long time, and I'm hoping that this hasn't already been asked and answered. The issue has to do with the parser's different behaviors WRT an IP address when you enter only the IP vs. what it does with the source IP during the parsing of a full spam.

For example, I received a false eBay message from an Adelphia broadband IP: [68.65.212.223]

When I first parsed the IP address by itself, before parsing the full spam, I saw a "[report history]" link and in the "Statistics" section, I also saw "68.65.212.223 listed in bl.spamcop.net (127.0.0.2)." This is what I would expect. So, I then parsed the full spam (with "Show technical details" turned on), and here's the Tracking URL:

http://www.spamcop.net/sc?id=z819216685z4d...3ce3625077879az

In the "Tracking message source: 68.65.212.223:" section, I see neither the "[report history]" for the IP *nor* the "listed in bl.spamcop.net" -- why not? This is not what I'd expect. I would expect the system to present me with the same information about that IP during a full parse as it does when parsing only the IP.

I've seen this behavior for a long time, and it's entirely consistent, as least in not displaying the "listed in bl.spamcop.net" information. I think this is either a bug, a simple programming omission, or a design flaw, and I'd like others to chime in with their opinions, and hopefully, if enough of us make noise, the "powers that be" will make the simple fix to the parsing system to show us that useful information that is currrently being withheld.

DT

Link to comment
Share on other sites

Item 1: Going to the way-back machine (this I think I do have docmented somewhere in here somewhere, but ..) the single=line look-up function and to paster=your=spam-in-the-box form were totally separate functions, totally separate data ebntry windows. Somewhere along the line, Julian coded in a cute little bit of code and removed the single-line look-ip entry "from" ... combining that input function into the paste-your-spam-in-the-box window ...

A single-line of code with no CR/LF at the end will be seen as a single-line input and will thusly jump over into that branch of code. If that first line of input text has a CR/LF at the end of it, the parse-the-spam branch is then taken. Those separate branches of code handle lots of things differently .. for example the many, many, many queries as to why the spam parser can't resolve a URL, but the single-line look-up will come back with results. All I can say is tha this is because of the differences in the code, which were also based on the different intended results and information.

This brings things up to a bit of dialog that I just had, working a bit of the same 'issue' ... but from a different perspective ... but I believe that this also has some connection to your query ... once again, looking at the differences in the code branches invloled and the intended purposes of the results ...

> > > I have no doubt that the easy answer is that everything is

> > > keyed to the IP, but .... why don't both queries return

> > > the same note?

> > >

> > > http://www.spamcop.net/sc?track=219.88.242.3

> > > http://www.spamcop.net/sc?track=loadbalancer1.orcon.net.nz

> > Because the system does not see them as the same thing. It records injection

> > reports against the IP which is why there is a report history altho there

> > are no reports in the database as the last one was greater than 90 days ago.

> > When you do the domain lookup it finds the IP and the reporting address but

> > indicates that there have been no reports against the domainname per se.  I

> > assume that is what you are asking?

> It was the 'internal database entry' that is/was of interest.

> ISP does not wish to receive reports regarding 219.88.242.3 - no date available

> only showing up in the IP query ...

>

> Richard forwarded a request from an ISP about deleting

> some Forum traffic ... there is an abundant amount of

> cluelessness offered up in that discussion, but one of

> the background things was the status of that 'note' ...

Right -- the reason that comment only showed up for the IP is because the

ISP turned off reports for the IP (which will stay off 24 hours or until the

first report after the 24 hr period) and there were reports for the IP at

some point.

More than you asked for, not quite answering your query specifically, but from the view of my systems analyst hat, all of the above makes sense to me <g> Different code, different intents, different results ... (picturing the "Document it???? Why, I know what it does!")

Link to comment
Share on other sites

I agree that both "[report history]" and "68.65.212.223 listed in bl.spamcop.net (127.0.0.2)

More Information.." should be a part of the Technical Details in a full Parse by the SpamCop Parsing and Reporting Service of spam implicating source IP Address 68.65.212.223, and similarly for any IP Address (that is, "report history" when warranted, and "listed" or "not listed" in bl.spamcop.net for every IP Address).

Link to comment
Share on other sites

(responding to Wazoo)

Your quoted conversation is *somewhat* related (more conceptually than in specifics), but I think it's a bit more "apples to organges" when compared to my query.

I'd like you to put my specific query to them, Wazoo, because I think it would be *very* useful for the system to both tell us that the source IP of the message we're parsing is already listed in the SCBL and also to present us with a History link for that IP, neither of which is currently happening. I'm not a programmer, but I've tinkered with enough code to guess that this wouldn't take very much effort to implement.

(responding to Jeff G.)

Thanks...that's what I'm looking for here...a groundswell of support so that we can get them to fix this. :-)

DT

Link to comment
Share on other sites

(responding to Jeff G.)

Thanks...that's what I'm looking for here...a groundswell of support so that we can get them to fix this.  :-)

34785[/snapback]

You're welcome. Please note that I did a bit of qualification and clarification via edit at the end of my previous Reply.
Link to comment
Share on other sites

(responding to Wazoo)

Your quoted conversation is *somewhat* related (more conceptually than in specifics), but I think it's a bit more "apples to organges" when compared to my query.

I'd like you to put my specific query to them, Wazoo, because I think it would be *very* useful for the system to both tell us that the source IP of the message we're parsing is already listed in the SCBL and also to present us with a History link for that IP, neither of which is currently happening. I'm not a programmer, but I've tinkered with enough code to guess that this wouldn't take very much effort to implement.

34785[/snapback]

Well, I'll agree that I did try to keep things on that "conceptual" level, but that's because getting into specifics would have drawn so much more into it .... the magic word of the day would be "server load" ... I'll agree that the code change might be minimal to add in yet another look-up. However, ..... somewhere in the past I remember trying to quantify some of things that happned after one hit the Submit button ... not wanting to recreate that effort here ... but the obvious bottom line would be a minimum of two more queries per IP address in analyzing of the spam, probably tapping into yet another computer that's holding that part of 'the' SpamCop database .... even though hardware has been upgraded, system count has increased, the current issues with "server load" are still present. Adding more queries into the analysis function, well .... things aren't quite so simple from that view ...

Sure, I'll be glad to send it up. However, you might want to take a look at the population of the KnowledgeBase thing I installed here. The working out of some issues on the "official" FAQ aren't going ike greased lightening, to say the least. I would suggest posting the request over in the New Feature Forum for starters so that the suggestion isn't lost due to being buried here ....

Link to comment
Share on other sites

However, you might want to take a look at the population of the KnowledgeBase thing I installed here.  The working out of some issues on the "official" FAQ aren't going ike greased lightening, to say the least.  I would suggest posting the request over in the New Feature Forum for starters so that the suggestion isn't lost due to being buried here ....

Will do, and thanks. I understand the issue of adding to the server load, but IMO (and hopefully others), the benefit of having that information available to the user outweighs the load issue. I'll post a compromise position in my "Feature Request."

[edit] I have now posted this issue as a Feature Request here: http://forum.spamcop.net/forums/index.php?showtopic=5210

DT

Link to comment
Share on other sites

I've seen this behavior for a long time, and it's entirely consistent, as least in not displaying the "listed in bl.spamcop.net" information. I think this is either a bug, a simple programming omission, or a design flaw, and I'd like others to chime in with their opinions, and hopefully, if enough of us make noise, the "powers that be" will make the simple fix to the parsing system to show us that useful information that is currrently being withheld.

34780[/snapback]

Sorry, but I can't get behind that.

The report history and Blocking List status are both easily available by opening a new browser window and querying the database directly, and asking the parse to display the Blocking List status would increase the queries on our BL database by something like two Million hits a day, which is definitely not something we want to do.

- Don D'Minion - SpamCop Admin -

Link to comment
Share on other sites

...asking the parse to display the Blocking List status would increase the queries on our BL database by something like two Million hits a day, which is definitely not something we want to do.

That's a shame...the omission of such basic data from the parse is a weakness. The workaround of opening a second window/tab is certainly an option, but a bit of an inconvenience to the user. Another possible compromise would be to offer this feature to users in the "Preferences" and set the default to "off." It could even be programmed to only display if the user actually uses the "Report spam" form manually (and only if the Preference has already been set to "on"), as opposed to being active during any batch or automatic processing. That would reduce its use, and the number of added queries to the BL would likely be insignificant if the function was limited in this fashion.

Please reconsider offering it as a non-default option for users.

(BTW...thanks for the speedy response!)

DT

Link to comment
Share on other sites

That's a shame...the omission of such basic data from the parse is a weakness.

The blocking list status is irrelevant to the parse. What would you do with the information if you had it? SpamCop's processing power has already been consumed by the time the parse gets to the point where it could tell you if the IP is listed.

- Don -

Link to comment
Share on other sites

The blocking list status is irrelevant to the parse.

OK, but then why display all the third-party BL info during the parse? Here's an example from a spam I just parsed:

219.136.233.176 not listed in dnsbl.njabl.org

219.136.233.176 not listed in dnsbl.njabl.org

219.136.233.176 not listed in cbl.abuseat.org

219.136.233.176 listed in dnsbl.sorbs.net ( 127.0.0.10 )

219.136.233.176 not listed in relays.ordb.org.

219.136.233.176 not listed in accredit.habeas.com

219.136.233.176 not listed in plus.bondedsender.org

219.136.233.176 not listed in iadb.isipp.com

Surely that also uses system resources? Why should the parsing system bother doing lookups in all those other systems (even if they are cached lookups)?

What would you do with the information if you had it?

Information is power. It would tell me if filtering that particular email account through a system using the SCBL would be useful, because if the IP was already "listed," then the spam would have been flagged/blocked/etc. by an SCBL-enabled system. Or, if I were reporting a spam hours after receiving it, it's possible that by then, the source IP would have made it into the SCBL due to other reports. This information would be useful to me in explaining the increasing amount of spams that are getting past the SC email account filters....and it *is* increasing.

It probably wouldn't affect whether or not I'd go ahead and complete the submission or not, but I would find the information useful, and I think others would also.

DT

Link to comment
Share on other sites

Sorry, I don't understand. It appears that every SpamCop Parse queries the servers serving (or internal systems mirroring) the dnsbl.njabl.org (twice?), cbl.abuseat.org, dnsbl.sorbs.net, and relays.ordb.org zones (at least until a hit is found) and some Parses query the servers serving (or internal systems mirroring) the accredit.habeas.com, plus.bondedsender.org, and iadb.isipp.com zones, and yet you can't afford similar queries to the servers serving (or internal systems mirroring) the SCBL and Report History?

Link to comment
Share on other sites

OK, but then why display all the third-party BL info during the parse?

We do the lookups for our own purposes and the parse always tells you what it's doing.

Information is power. It would tell me if filtering that particular email account through a system using the SCBL would be useful, because if the IP was already "listed," then the spam would have been flagged/blocked/etc. by an SCBL-enabled system. Or, if I were reporting a spam hours after receiving it, it's possible that by then, the source IP would have made it into the SCBL due to other reports. This information would be useful to me in explaining the increasing amount of spams that are getting past the SC email account filters....and it *is* increasing.

The bottom line here is that I think there wouldn't be more than ten people who would even realize the information was available, much less figure out a way to take advantage of it. The vast majority of users don't even review the technical details of the parse.

I'm not going to ask the development engineers to write code for something I can't personally support.

- Don D'Minion - SpamCop Admin -

Link to comment
Share on other sites

The vast majority of users don't even review the technical details of the parse.

That's not surprising...who has time? Thanks for giving this some consideration, Don, and for your speedy responses. The "extra window" workaround will have to suffice for now, and I have referenced it in the How to use .... > SpamCop Reporting forum in the existing Topic titled Steps taken by the parser, A detailed view of what the parser does (and what the parser does NOT do...) :-)

DT

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...