Spam not parsing - source IP missing?

[Tim P]

http://www.spamcop.net/sc?id=z754553313z3f...4ec2f8a6241859z

spam from this outfit has always parsed correctly, until this one.

error (relevant parsing lines shown):

.

4: Received: from 216.171.217.252 (EHLO ns1.eprosender.com) (216.171.217.252) by mta168.mail.re2.yahoo.com with SMTP; Wed, 20 Apr 2005 06:00:33 -0700

Hostname verified: ns1.eprosender.com

Trusted site mailgate.cesmail.net received mail from 216.171.217.252

5: Received: from ns1.eprosender.com (localhost.eprosender.com [127.0.0.1]) by ns1.eprosender.com (8.12.10/8.12.7) with ESMTP id j3KCurqJ082519 for <x>; Wed, 20 Apr 2005 05:56:53 -0700 (PDT) (envelope-from nate[at]ns1.eprosender.com)

Internal handoff or trivial forgery

No source IP address found, cannot proceed.

Add/edit your mailhost configuration

Finding full email headers

Submitting spam via email (may work better)

Example: What spam headers should look like

Nothing to do.

Wrong....216.171.217.252 has ALWAYS parsed as the source of this spam. What changed?

Upon further review:

http://mailsc.spamcop.net/mcgi?action=show...id;val=44082974

shows all successful reports, some of which were mine.

[turetzsr]

...You may want to try again ... it just worked for me!

[SpamCopAdmin]

New code we put into production today caused some parse failures for users with Mailhosts configured.

The problem has been fixed and the new code published.

Sorry for the freakout!

- Don -

[Tim P]

New code we put into production today caused some parse failures for users with Mailhosts configured.

The problem has been fixed and the new code published.

Sorry for the freakout!

26728[/snapback]

Parsed at this time,

thanks.

[nomorespam]

Hmm... Maybe something's still a bit broke? This is the first time this has happened for me.

http://www.spamcop.net/sc?id=z754650523z57...9909ad48724719z

Here are the relevant lines:

1: Received: from unknown (HELO vtoy.fi) ([at]211.229.225.71) by linus.fmls.ca with SMTP; 21 Apr 2005 07:22:36 -0000

No unique hostname found for source: 211.229.225.71

IslandTech secondary received mail from sending system 211.229.225.71

2: Received: from 158.239.44.37 by smtp.espoo.fi; Thu, 21 Apr 2005 07:18:27 +0000

No unique hostname found for source: 158.239.44.37

Possible forgery. Supposed receiving system not associated with any of your mailhosts

Will not trust anything beyond this header

No source IP address found, cannot proceed.

Add/edit your mailhost configuration

Finding full email headers

Submitting spam via email (may work better)

Example: What spam headers should look like

Nothing to do.

It would seem 211.229.225.71 is identified as the source, but after the next fake Received line, SpamCop appears to have forgotten. Any ideas?

[Wazoo]

I'd suggest you contact the following and see if they can kick their server around a bit. Header data contents are broken. Your Line 1 is not a valid construct so the only IP left is your ISP's server.

04/21/05 05:36:21 IP block 142.179.102.53

Trying 142.179.102.53 at ARIN

Trying 142.179.102 at ARIN

OrgName: Stentor National Integrated Communications Network

OrgID: STEN

Address: 110 O'Connor St., Floor 3

City: Ottawa

StateProv: ON

PostalCode: K1P-IH1

Country: CA

NetRange: 142.179.0.0 - 142.179.255.255

CIDR: 142.179.0.0/16

NetName: STENTOR21

NetHandle: NET-142-179-0-0-1

Parent: NET-142-0-0-0-0

NetType: Direct Assignment

NameServer: NANO.BC.TAC.NET

NameServer: PICO.BC.TAC.NET

Comment:

RegDate: 1992-08-27

Updated: 2002-08-28

AbuseHandle: TEL1256-ARIN

AbuseName: TELUS Communications

AbusePhone: +1-604-444-5791

AbuseEmail: abuse[at]telus.com

TechHandle: PSINET-CA-ARIN

TechName: TELUS Communications Inc.

TechPhone: +1-613-780-2200

TechEmail: swip[at]swip.ca.telus.com

OrgTechHandle: ZS74-ARIN

OrgTechName: Stentor National IntegratedCommunications Network

OrgTechPhone: +1-613-781-9095

OrgTechEmail: stentornet.admin[at]bell.ca

[nomorespam]

Something has definitely changed with SpamCop. I took some spam I filed April 14th (http://www.spamcop.net/mcgi?action=gettrac...rtid=1402907806) without any issues then, and re-parsed it just now and got the same error as above.

Since Spamcop is actually identifying the correct bits from the recevied line, why do you say the header data is broken? It would seem to me if Spamcop is finding what it needs, it should use it.

If the header is (now) considered broken to Spamcop, then the message should say something like "This header is broken, ignoring it" rather than "IslandTech secondary received mail from sending system 211.229.225.71". At best, it's misleading how it works now.

Here's another spam I just received and parsed with the same problem: http://www.spamcop.net/sc?id=z754879538z3c...cac5c0a3a9fb8bz

Just my 2 cents worth.

[emboehm]

I'm seeing the same things as well

http://www.spamcop.net/sc?id=z755087287ze0...e9b47d72920919z

Parsing header:

0: Received: from ms-mta-03-eri0 (ms-mta-03-eri0 [10.25.8.236]) by ms-mss-05.southeast.rr.com (iPlanet Messaging Server 5.2 HotFix 2.04 (built Feb 8 2005)) with ESMTP id <0IFC009QFAUXXS[at]ms-mss-05.southeast.rr.com>; Fri, 22 Apr 2005 04:42:33 -0400 (EDT)

Internal handoff at RoadRunner

1: Received: from ncmx03.mgw.rr.com (ncmx03.mgw.rr.com [24.25.4.97]) by ms-mta-03.southeast.rr.com (iPlanet Messaging Server 5.2 HotFix 2.04 (built Feb 8 2005)) with ESMTP id <0IFC009OHAUW2S[at]ms-mta-03.southeast.rr.com>; Fri, 22 Apr 2005 04:42:33 -0400 (EDT)

Hostname verified: ncmx03.mgw.rr.com

RoadRunner received mail from RoadRunner ( 24.25.4.97 )

2: Received: from 24.25.4.97 ([219.131.145.217]) by ncmx03.mgw.rr.com (8.12.10/8.12.8) with SMTP id j3M8fFTo008490; Fri, 22 Apr 2005 04:42:21 -0400 (EDT)

No unique hostname found for source: 219.131.145.217

RoadRunner received mail from sending system 219.131.145.217

No source IP address found, cannot proceed.

[turetzsr]

I'm seeing the same things as well

http://www.spamcop.net/sc?id=z755087287ze0...e9b47d72920919z

<snip>

No source IP address found, cannot proceed.

26822[/snapback]

...Interesting: I try pinging the last IP address (219.131.145.217) and get timeout. TRACERT does 28 hops, then timeout. Geektools Whois show the following for this IPA:

Final results obtained from whois.apnic.net.

Results:

% [whois.apnic.net node-2]

% Whois data copyright terms http://www.apnic.net/db/dbcopyright.html

inetnum: 219.128.0.0 - 219.137.255.255

netname: CHINANET-GD

descr: CHINANET Guangdong province network

descr: Data Communication Division

descr: China Telecom

country: CN

admin-c: CH93-AP

tech-c: IC83-AP

mnt-by: MAINT-CHINANET

mnt-lower: MAINT-CHINANET-GD

status: ALLOCATED NON-PORTABLE

changed: hostmaster[at]ns.chinanet.cn.net 20020424

changed: hm-changed[at]apnic.net 20041207

source: APNIC

person: Chinanet Hostmaster

address: No.31 ,jingrong street,beijing

address: 100032

country: CN

phone: +86-10-66027112

fax-no: +86-10-58501144

e-mail: hostmaster[at]ns.chinanet.cn.net

e-mail: anti-spam[at]ns.chinanet.cn.net

nic-hdl: CH93-AP

mnt-by: MAINT-CHINANET

changed: hostmaster[at]ns.chinanet.cn.net 20021016

remarks: hostmaster is not for spam complaint,please send spam complaint to anti-spam[at]ns.chinanet.cn.net

source: APNIC

...Hope this helps someone more knowledgeable than I....

[Wazoo]

emboehm - your Tracking URL - http://www.spamcop.net/sc?id=z755087287ze0...e9b47d72920919z

mine without MailHost involved - http://www.spamcop.net/sc?id=z755106005zbf...2833fd3944efb2z

nomorespam - your last - http://www.spamcop.net/sc?id=z754879538z3c...cac5c0a3a9fb8bz

mine without MailHost - http://www.spamcop.net/sc?id=z755108083z5c...75f8cf533b219ez

(Noting that the data changed a bit from your first example)

In both cases, the MailHost thing stumbles over trying not to self-report (my guess)

turetzsr - those probes can be killed off by turning off/rejecting/blocking ICMP traffic, usually with a firewall or router

Note kicked back to Don .....

[nomorespam]

Just parsed another spam, but got a different error message:

hsia.telus.net does not report source IP correctly

No source IP address found, cannot proceed.

This is definitely an improvement in so much the error message is matching the logic, but I still can't report spam.

The site in question is our secondary mx which is where 99% of our spam comes through, so I'd like to be able to file it. That site uses qmail. Sendmail is our primary mx. Does qmail just make "broken" headers, or is there a configuration we can make to qmail so Spamcop will like the headers?

I'm curious to find out why this change was made to Spamcop's parser. We weren't having any trouble with how Spamcop parsed email that travelled through that path before, but now we have a 100% failure rate. The IP Spamcop identified as the source is the one I'd pick to report manually if Spamcop was out of the equation.

Is it possible to soften the header parsing rules (or (gulp) make an exception for qmail) so we can report spam, or are we left out in the spammy cold?

Thanks for any assistance.

[Wazoo]

Not 100% sure of "your changed data" as this 'new' stuff is the same as I provided in my previous posts - reference "mine without MailHost" ... you didn't provide a Tracking URL here.

The error condition suggests that a configuration change is needed, such that the header lines are inserted correctly.

Julian rarely makes known what changes have been made to his source code, recall that he's working against all the spammers in the world that are attempting to outwit the parser .. so all that can be said is that Don posted that there was something wrong with some code that then got worked on some more. That said, no response to my last e-mail, other than your statement that something changed. (which is usually how it works, Julian sees/hears of a problem, he makes changes, and the problem is then noticed by its absence. Today's focus would have been bringing the system back on-line.)

[emboehm]

New error today

http://www.spamcop.net/sc?id=z755532057z99...9d2282cd746fe0z

Parsing header:

0: Received: from ms-mta-01-eri0 (ms-mta-01-eri0 [10.25.8.234]) by ms-mss-05.southeast.rr.com (iPlanet Messaging Server 5.2 HotFix 2.04 (built Feb 8 2005)) with ESMTP id <0IFD00LNQLZ9O2[at]ms-mss-05.southeast.rr.com> for x; Fri, 22 Apr 2005 21:40:21 -0400 (EDT)

Internal handoff at RoadRunner

1: Received: from ncmx03.mgw.rr.com (ncmx03.mgw.rr.com [24.25.4.97]) by ms-mta-01.southeast.rr.com (iPlanet Messaging Server 5.2 HotFix 2.04 (built Feb 8 2005)) with ESMTP id <0IFD00KJRLZ967[at]ms-mta-01.southeast.rr.com> for x (ORCPT x); Fri, 22 Apr 2005 21:40:21 -0400 (EDT)

Hostname verified: ncmx03.mgw.rr.com

RoadRunner received mail from RoadRunner ( 24.25.4.97 )

2: Received: from standish59.freeserve.co.uk ([222.237.28.249]) by ncmx03.mgw.rr.com (8.12.10/8.12.8) with SMTP id j3N1e94E008877 for <x>; Fri, 22 Apr 2005 21:40:18 -0400 (EDT)

ncmx03.mgw.rr.com does not report source IP correctly

No source IP address found, cannot proceed.

According to whois, 222.237.28.249 is from Hanaro Telecom in Korea

[nomorespam]

2: Received: from standish59.freeserve.co.uk ([222.237.28.249]) by ncmx03.mgw.rr.com (8.12.10/8.12.8) with SMTP id j3N1e94E008877 for <x>; Fri, 22 Apr 2005 21:40:18 -0400 (EDT)

ncmx03.mgw.rr.com does not report source IP correctly

No source IP address found, cannot proceed.

According to whois, 222.237.28.249 is from Hanaro Telecom in Korea

26909[/snapback]

Other than the above quoted line not being in the line #0 position, how is it substantially any different from this:

0: Received: from cwc.com.au ([216.62.211.179]) by sven.islandtech.bc.ca (8.11.6/8.11.6) with SMTP id j22NFPu21129 for <x>; Wed, 2 Mar 2005 15:15:27 -0800

No unique hostname found for source: 216.62.211.179

IslandTech primary received mail from sending system 216.62.211.179

Tracking url: http://www.spamcop.net/sc?id=z755543807zef...481da817adef32z which parses just fine? In this case cwc.com.au resolves to 203.30.164.4, and 216.62.211.179 is SBC Internet in Texas.

I doubt Spamcop would try to match the provided hostname (from the sender) with the IP address that connected to the receiver. That hostname is almost always spoofed.

To Wazoo: As for

nomorespam - your last - http://www.spamcop.net/sc?id=z754879538z3c...cac5c0a3a9fb8bz

mine without MailHost - http://www.spamcop.net/sc?id=z755108083z5c...75f8cf533b219ez

(Noting that the data changed a bit from your first example)

The only difference I can figure is that because you don't use my mailhost config, Spamcop needs to do the chain test (among others for validity) instead of simply making sure the handoff is to a configured mailhost.

[StevenUnderwood]

Other than the above quoted line not being in the line #0 position, how is it substantially any different from this:

26912[/snapback]

First case:

ncmx03.mgw.rr.com does not report source IP correctly

No source IP address found, cannot proceed.

Either spamcop does not like the form used by the receiving server (though the form is accepted elsewhere) or the IP is known to have problems. Either way is will not trust headers from ncmx03.mgw.rr.com.

Second case:

No unique hostname found for source: 216.62.211.179

IslandTech primary received mail from sending system 216.62.211.179

The first line is just a warning that 216.62.211.179 does not resolve back to cwc.com.au but it does NOT stop parsing. It accepts the IP address and goes onto report that IP as the source.

Basically, the difference is, it trusts one servers answers, but not the others, possibly due to previous parses or problems.

[SpamCopAdmin]

hsia.telus.net does not report source IP correctly

ncmx03.mgw.rr.com does not report source IP correctly

Those lines from the parse tell us that the server has been caught recording a forged or non-routeable IP as the source and has been flagged as untrustworthy. The parse will not accept the source IP recorded by servers marked as Liars.

In the case of 142.179.102.53 = s142-179-102-53.bc.hsia.telus.net, the entire range of servers using the hsia.telus.net naming convention have been flagged as Liars.

Since the 'lying' server is being tagged as the source of the spam, and it is registered as the user's host, the parse fails and processing stops.

The problem now becomes whether or not to remove the Liar status from those servers. My thinking is that since the user was able to register a Host for that network, the servers are probably correctly recording the source IP, and I should remove the flag.

On the other hand, if I'm wrong, it lays us wide open to accepting spammer forgeries. Just because the server that handled our Mailhost test probe correctly recorded the source IP, doesn't mean the other mail handlers in the network are properly configured.

I removed the 'Liar" flag from those two servers. Now we wait to see if anything bad happens.

Users with Mailhosts configured who are still experiencing the parse failures because the parse wants to go after their own host should contact me directly for a review. Please send the Tracking URL. It's the only way for me to see what SpamCop saw.

service[at]admin.spamcop.net

- Don -

[Farelf]

Just to add a snippet - it is apparently possible for an IP address in the configured mail hosts to be flagged as both trusted and untrusted at the same time. This leads to the above symptoms/messages for some spam (consistently, holds up after repeated refreshes) yet others with the same routing will go through without problem (again consistently). The resolution is to contact Don (service[at]admin.spamcop.net) with the detail including tracking URL. This would be rare, but a little weird to behold when it happens. Thanks to Don for sorting out "my" case in a trice.