Sign in to follow this  
Followers 0
SpamCopAdmin

IPv6 Routing Support

84 posts in this topic

The "Right" way of doing this would have been to add the IPv6 detection code, and have it log to a file for debugging. This doesn't change the existing behavior from the user perspective.

Since you know the spamcop code so well, why don't you write your own parser with IPv6 working and we can all use your code and leave SPamCop behind.

Sorry, but I am tired of everyone coming in here "knowing" how easy it would be to fix this problem when they have no idea what the code looks like, code which was self-described by the original developer as "spaghetti".

Share this post


Link to post
Share on other sites

...Steven, while I share your frustration and agree with your point, I think it may be misplaced to direct it at techie. IIUC, techie is making a suggestion about how to code for IPv6 in general, not how specifically to do it in SpamCop and has made no public judgment about whether the time it has taken is appropriate. On the other hand, techie's "The wrong way is to simply add a check for IPv6, and puke if found, which is what currently happens, as of the update in March 2011" comment seems to assume something that isn't true -- that the current code was intended to address IPv6, which (IIUC) it was not!

Share this post


Link to post
Share on other sites

The "Right" way of doing this would have been to add the IPv6 detection code, and have it log to a file for debugging. This doesn't change the existing behavior from the user perspective.

The detection code should include checks for the IPv6 loopback address [::1], and treat it the same as 127.0.0.1.

We should face the facts that IPv6 is not human, nor code friendly. IPv4 was easy to code for because it HAD three periods. IPv6 can have any number of colons, but not more than eight. Code that matches IPv6 will always be complex and never as simple as IPv4, as seen below. I suspect this is partly why SpamCop has not fully implemented it yet.

m/^([0-9A-Fa-f]{1,4}:){1,7}([0-9A-Fa-f]{1,4}){1,1}|([0-9A-Fa-f]{1,4}:){1,6}(:[0-9A-Fa-f]{1,4}){1,1}|([0-9A-Fa-f]{1,4}:){1,5}(:[0-9A-Fa-f]{1,4}){1,2}|([0-9A-Fa-f]{1,4}:){1,4}(:[0-9A-Fa-f]{1,4}){1,3}|([0-9A-Fa-f]{1,4}:){1,3}(:[0-9A-Fa-f]{1,4}){1,4}|([0-9A-Fa-f]{1,4}:){1,2}(:[0-9A-Fa-f]{1,4}){1,5}|([0-9A-Fa-f]{1,4}:){1,1}(:[0-9A-Fa-f]{1,4}){1,6}$/

...Steven, while I share your frustration and agree with your point, I think it may be misplaced to direct it at techie. IIUC, techie is making a suggestion about how to code for IPv6 in general, not how specifically to do it in SpamCop and has made no public judgment about whether the time it has taken is appropriate. On the other hand, techie's "The wrong way is to simply add a check for IPv6, and puke if found, which is what currently happens, as of the update in March 2011" comment seems to assume something that isn't true -- that the current code was intended to address IPv6, which (IIUC) it was not!

These are some generic questions, but I believe they get to the root of the problem. How do you go about reporting IPv6 to the upsteam, which is to say the ISP? How do you add the IPv6 address to a block list?

I believe the main reasons for the delay is that they do not need to just match IPv6, but they also need to get other underlying code updated as well. We know that they are able to find the IPv6 address now, because the page says it found IPv6 and stops. I believe that SpamCop is working on the whois, reverse DNS, blacklisting servers, and also working with the abuse.net DB to get all of it IPv6 compatible. SpamCop needs to get all of their code updated so it handles IPv6 in all of the code, not just the detector portion.

Edited by gnarlymarley

Share this post


Link to post
Share on other sites

We should face the facts that IPv6 is not human, nor code friendly. IPv4 was easy to code for because it HAD three periods. IPv6 can have any number of colons, but not more than eight. Code that matches IPv6 will always be complex and never as simple as IPv4, as seen below. I suspect this is partly why SpamCop has not fully implemented it yet.

m/^([0-9A-Fa-f]{1,4}:){1,7}([0-9A-Fa-f]{1,4}){1,1}|([0-9A-Fa-f]{1,4}:){1,6}(:[0-9A-Fa-f]{1,4}){1,1}|([0-9A-Fa-f]{1,4}:){1,5}(:[0-9A-Fa-f]{1,4}){1,2}|([0-9A-Fa-f]{1,4}:){1,4}(:[0-9A-Fa-f]{1,4}){1,3}|([0-9A-Fa-f]{1,4}:){1,3}(:[0-9A-Fa-f]{1,4}){1,4}|([0-9A-Fa-f]{1,4}:){1,2}(:[0-9A-Fa-f]{1,4}){1,5}|([0-9A-Fa-f]{1,4}:){1,1}(:[0-9A-Fa-f]{1,4}){1,6}$/

These are some generic questions, but I believe they get to the root of the problem. How do you go about reporting IPv6 to the upsteam, which is to say the ISP? How do you add the IPv6 address to a block list?

I believe the main reasons for the delay is that they do not need to just match IPv6, but they also need to get other underlying code updated as well. We know that they are able to find the IPv6 address now, because the page says it found IPv6 and stops. I believe that SpamCop is working on the whois, reverse DNS, blacklisting servers, and also working with the abuse.net DB to get all of it IPv6 compatible. SpamCop needs to get all of their code updated so it handles IPv6 in all of the code, not just the detector portion.

Yes, and considering all the ways to abbreviate an IPv6 address, just converting the address-as-text (as found in the mail headers) to the address-as-128-bits (the unique value which can be meaningfully compared for equality, and so blacklisted or not) is an unobvious task, albeit well-defined.

At least now (and unlike what happened when the first IPv6 spam appeared), when SpamCop chokes on an IPv6 spam it gives a clear message that it currently doesn't support IPv6. One could always wish for full IPv6 support; but let's not forget that even after the purchase by IronPort and the latter's purchase by Cisco, SpamCop is still cruelly understaffed and, AFAIK, the only person who more or less masters the SpamCop source is Julian, and I'm not sure how much time he can dedicate to finding (in the “mass of spaghetti†mentioned a few posts ago, and without neglecting his other duties) the relevant code for this particular problem (which probably is neither the only problem nor the most urgent one), fixing it, testing it, checking for side-effects, etc. etc. etc.

I don't know about y'all, but the IPv6 spam messages that fall into my inbox are (still) few and far between; I believe that we still have several years before they become the majority, and I'm confident that the SpamCop code will be suitably modified in time to handle them satisfactorily before they become an unbearable nuisance.

Share this post


Link to post
Share on other sites

Good news to all the IPv6 folks out there.

http://www.spamcop.net/sc?id=z5324131362z4...47234ee1c0da1fz

There is some stuff still lacking like this tracking link I found couldn't find a person to report to, but that should be a minor fix. It is something about "Cannot find ip range in whois output".

turetzsr, I think you have connections with Julian. Can you pass on the congratulations?

Share this post


Link to post
Share on other sites
<snip>

turetzsr, I think you have connections with Julian. Can you pass on the congratulations?

...Actually, no, everyone here has the same connections with SpamCop staff that I have -- you can just send an e-mail to deputies[at]admin.spamcop.net and a SpamCop deputy will (hopefully) let the relevant folks know.

Share this post


Link to post
Share on other sites

Looks like work continues - gnarlymarley's link just at the moment has the "Unable to process header. IPv6 addresses are not supported.

No source IP address found, cannot proceed." lines.

<!-- SpamCop::Web::Look $Revision: #15 $ produced by prod-sc-www1 -->

See http://forum.spamcop.net/forums/index.php?...amp;#entry80735

The analysis should find (I guess):

2a02:8420:4e46:2e00:211:32ff:fe11:32f5

tracks to SFR-BROADBAND-USER (FR)

inet6num: 2a02:8420::/32

...

remarks: ***********************************

remarks: * Abuse e-mail: abuse[at]gaoland.net *

remarks: ***********************************

...

role: LDCOM Legal Contact

...

abuse-mailbox: abuse[at]gaoland.net

...

role: LDCOM Networks Tech Contact

...

abuse-mailbox: abuse[at]gaoland.net

...

% This query was served by the RIPE Database Query Service version 1.8.13 (WHOIS2)

Share this post


Link to post
Share on other sites

Hm, yes, trying to re-parse with that same tracking URL gives me the same errors, but SC also tells me that "reports have already been sent" — albeit to nomaster[at]devnull.spamcop.net — concerning that IPv6 address.

Parsing header:

Received: from [2a02:8420:4e46:2e00:211:32ff:fe11:32f5] (helo=favre.fr) by kaysville.yaritz.net with esmtp (Exim 4.66 (FreeBSD)) (envelope-from <admin[at]megaleecher.net>) id 1SSIGO-0007ol-Dg for me[at]yaritz.net; Wed, 09 May 2012 19:33:29 -0600

Unable to process header. IPv6 addresses are not supported.

No source IP address found, cannot proceed.

Add/edit your mailhost configuration

Finding full email headers

Submitting spam via email (may work better)

Example: What spam headers should look like

Reports regarding this spam have already been sent:

Re: 2a02:8420:4e46:2e00:211:32ff:fe11:32f5 (Administrator of network where email originates)

Reportid: 5765414954 To: nomaster[at]devnull.spamcop.net

______________________________________________________________________

Report another spam?

Share this post


Link to post
Share on other sites

Yes Tony - trial, some IPv6 spam parsed, bugs to be fixed so new version retracted, fixed version to be tried next week as advised in that other post by email_support. There was an opportunity while the new version was operating to add a report (if a paying user) to the correct abuse desk, if you were able to determine it. I use IPNetInfo.exe with the "Detect IPv6 addresses" box checked for those determinations since I am congenitally confused when it comes to using RIPE lookups unaided :D

Steve

Share this post


Link to post
Share on other sites

Adding another e-mail address to my account I discovered this error:

Sorry, SpamCop has encountered errors:

Headers not found.

It appears you did not provide all email headers. Please consult this FAQ

for more information on getting full headers from your software, then try

again.

Could this also be an effect of IPv6 addresses in the headers, because I'm sure I've included all headers.

I hope the IPv6 routing support will also include support for setting up e-mail addresses and Mail eXchanges that have and therefor by default use IPv6.

Share this post


Link to post
Share on other sites

Adding another e-mail address to my account I discovered this error:...

You were amending your mailhosting on a reporting account? Write to SpamCop Admin (Don D'Minion) with all the details:

service[at]admin.spamcop.net

He will know the answer and may(??) be able to help sort you out.

Share this post


Link to post
Share on other sites

...Steven, while I share your frustration and agree with your point, I think it may be misplaced to direct it at techie. IIUC, techie is making a suggestion about how to code for IPv6 in general, not how specifically to do it in SpamCop and has made no public judgment about whether the time it has taken is appropriate. On the other hand, techie's "The wrong way is to simply add a check for IPv6, and puke if found, which is what currently happens, as of the update in March 2011" comment seems to assume something that isn't true -- that the current code was intended to address IPv6, which (IIUC) it was not!

IIRC, the Spamcop update of March 2011 was advertised as providing for future support for IPv6.

Immediately after this update, I started receiving bounces for spam containing IPv6 loopback addresses, which had never been rejected prior to this update. I went back through my spam archives, and verified that I had submitted spam containing IPv6 loopback addresses several time during the previous month, none of which had been rejected.

My assumption, based on observed Spamcop behavior before and after that update, is that code to detect IPv6 was added during that update, and that instead of using that code to gather data, or as a step-by-step sanity check, it was instead used as a cop-out.

My comments on the "right way" and the "wrong way" are based on one assumption about the parsing process, namely that the process works from the recipient back towards the sender.

This is the logical method, since Spamcop needs to eliminate hosts within your MX path, so as not to report your receiving server as the spam source, which of course it is not.

The host immediately upstream of your receiving server is the last host which can be properly verified, since all headers upstream of that point can be forged. If Spamcop trusts the header upstream of that point, it is only because Spamcop trusts the upstream ISP (such as Yahoo/Google/AOL/etc), and has decided that the internal headers added by those ISP's are trustworthy.

Given that the parsing process needs to work backwards, it is logical that sanity check for IPv6 headers should be done at each step in the parsing process, which would eliminate all of the problems with IPv6 loopback addresses in the first hop, forged IPv6 headers, IPv6 tunnels and 6-to-4 gateways prior to the upstream server. This would not eliminate the problems with IPv6 use at the receiving site, or actual IPv6 transit between the receiving server and the host immediately upstream, but it would eliminate the vast majority of the current set of problems, since the spam source would be reached, and parsing terminated, prior to an IPv6 address being encountered.

Instead, what I have observed is that the parser looks for IPv6 addresses anywhere in the header as the first step, and pukes if any IPv6 addresses are found, instead of actually attempting to parse the headers in proper sequence, and only puking if an IPv6 address is encountered prior to determining the spam source.

Edited by techie

Share this post


Link to post
Share on other sites
<snip>

My comments on the "right way" and the "wrong way" are based on one assumption about the parsing process, namely that the process works from the recipient back towards the sender.

<snip>

...The assumption is, from what I understand, correct. What we don't know is whether that fact allows us to conclude that it is necessarily feasible and "easy" for the parser to ignore all IPv6 addresses that appear prior to the point it decides that it has found the location at which it can stop trusting the headers. I think that is not an unreasonable conclusion but it is, nevertheless, a conclusion the validity of which we will never know because the SpamCop engineers won't ever reveal the answer to us. And since it's their product, they get to define the rules, so I for one feel unworthy to challenge it and can't imagine that anyone else outside the SpamCop staff circle is able to do so for certain, either.

Share this post


Link to post
Share on other sites

...The assumption is, from what I understand, correct. What we don't know is whether that fact allows us to conclude that it is necessarily feasible and "easy" for the parser to ignore all IPv6 addresses that appear prior to the point it decides that it has found the location at which it can stop trusting the headers. I think that is not an unreasonable conclusion but it is, nevertheless, a conclusion the validity of which we will never know because the SpamCop engineers won't ever reveal the answer to us. And since it's their product, they get to define the rules, so I for one feel unworthy to challenge it and can't imagine that anyone else outside the SpamCop staff circle is able to do so for certain, either.

Well, if we also assume two other things, namely that the IPv6 detection code is written as a subroutine, and that the parsing process works on the headers one hop at a time, then the real question is: why did they call the subroutine at the beginning, instead of as a conditional during the parsing process?

Share this post


Link to post
Share on other sites

Anyone know the ETA on IP V6 reporting? Even guesstimate would be nice. I have a couple nice, juicy spams waiting to report, but SC wont' accept them because they are IP V6. :(

Share this post


Link to post
Share on other sites

I guess if I was getting a number of them I would be tempted to manually report - but, in general, that would be an irresponsible thing to recommend to the general membership and, with most such reports ignored (and some maybe ending up in the hands of the spammers), fairly pointless or dangerous. Yes, it will be great to get the ease and security of the parser and reporting system in train on these, and the leverage of the SCbl behind it.

Can't help but reflect that some of what was learned in the last attempt will be forgotten with the passage of time - nobody documents so well that it could be avoided. Oh well, obviously that will be more than offset by what the engineers' community has learned in the interim, when the next attempt succeeds - as it must.

Share this post


Link to post
Share on other sites

The next SpamCop version supports IPv6 parsing.

The release should be out within the next 30 days. No promises. It could easily be 60 days, or even 90 days, but we're hopeful that it will be sooner rather than later.

- Don D'Optimistic - SpamCop Admin -

- Service[at]Admin.SpamCop.net -

.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0