Spamcop stripping subject headers - ATT: SC ADMIN

[jrsherrard]

First, forgive my near complete ignorance in these matters.

For the second time in the last six months, my email has been stripped of subject headers, which are dumped into the body of the message. It happens on my ISPs end, but they don't really understand why. Here's a portion of my relevant correspondance:

"On Oct 9, 2004, at 1:04 PM, Dan Pewzner wrote:

Hi Jean,

Sorry about that. It happens when I route email through one of our

mail servers here. I don't understand why it only happens to spamcop.net

mail though.

Its very odd-- if I have your email go from a qmail-1.03

server to a sendmail (8.12.8) server, somehow spamcop doesn't like the headers.

No other servers have any problems. Thanks, Dan"

Any suggestions? Solutions I can relay to my ISP?

Thanks--Jean

[Wazoo]

Not much to say from this side of the screen, especially with no data to go on, no examples, no tools-in-use description .. and if your ISP has the "problem" tracked down to a single server, I'm not sure how you'd expect someone"here" to advise someone "there" about just what "they" did do configure this one server to be different than all their other servers. You may use the "complete ignorance" excuse as far as not understanding these things, but I have to point out that you didn't offer enough data to allow anyone here to work in a mode beyond "complete ignorance" either.

[jrsherrard]

Not much to say from this side of the screen, especially with no data to go on, no examples, no tools-in-use description .. and if your ISP has the "problem" tracked down to a single server, I'm not sure how you'd expect someone"here" to advise someone "there" about just what "they" did do configure this one server to be different than all their other servers.  You may use the "complete ignorance" excuse as far as not understanding these things, but I have to point out that you didn't offer enough data to allow anyone here to work in a mode beyond "complete ignorance" either.

18575[/snapback]

Reading between the lines, I must assume that this is an experience entirely unique to me. In other words, mangled headers have never occurred before at spamcop. Unless I include the first time it happened to me, several months ago.

Again, no subject indicated in these messages.

Here, for example, is what the message body looks like:

From paul[at]dorpat.com Sat Oct 09 03:42:37 2004

Return-Path: <paul[at]dorpat.com>

Received: (qmail 4982 invoked from network); 8 Oct 2004 20:42:37 -0700

Received: from unknown (HELO dorpat.com) (63.249.15.84)

by mail.zipcon.net with SMTP; 8 Oct 2004 20:42:37 -0700

Date: Fri, 8 Oct 2004 20:39:56 -0700

Mime-Version: 1.0 (Apple Message framework v553)

Content-Type: text/plain; delsp=yes; charset=US-ASCII; format=flowed

Subject: Good pics of road - looks like Afganastan in these

From: Paul Dorpat <paul[at]dorpat.com>

To: Jean Sherrard <sherrard[at]zipcon.net>

Content-Transfer-Encoding: 7bit

Message-Id: <E3AF23E0-19A4-11D9-A61A-0003937007C2[at]dorpat.com>

X-Mailer: Apple Mail (2.553)

X-spam-Level:

X-spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on sim.zipcon.net

X-spam-Status: No, hits=-4.0 required=3.0 tests=AWL,BAYES_00 autolearn=ham

version=2.64

images.google.com/imgres?imgurl=http://www.discoweb.org/quilomene/

Q9.jpg&imgrefurl=http://www.discoweb.org/quilomene/

&h=600&w=400&sz=76&tbnid=BbClBK1u2uIJ:&tbnh=132&tbnw=88&start=14&prev=/

images%3Fq%3Dcolockum%2B%26hl%3Den%26lr%3D%26ie%3DUTF-8%26sa%3DG

[Wazoo]

Once again, what you have offered actualy supplies little data. How does your sample correlate to "the screwed up server" ... you call it "the body" but I/we have no idea if you know what the definition of "body" actually means "here" .... There's no way to really get into your sample and how the parser handles is because of the nature of this Forum software and (again) whatever tools-are-in-use at your end ...

From the top .. submit your spam, capture and post the Tracking URL of one of the failed itesm .. define what you are using at your end, define how you are submitting your spam .... read some of the other discussions that exist here to see what and how a successful trouble-shooting technique includes.

[StevenUnderwood]

Wazoo, this is NOT a reporting issue but a mail handling issue. The OP believes it HAS to be spamcop causing the problems.

Looking at your old post, I remember this issue, but you never replied there to tell us the solution, otherwise you could have found it yourself.

I am not experiencing problems and these forums would be full if others were experiencing problems. You are the only one having problems so lets start with your configuration.

You never indicated (in any post) how you get your email to spamcop. Do you have spamcop POP the messages or do you forward from your ISP (with the 3 other accounts)?

You also don't indicate how you retreive your messages. Do you POP or IMAP them directly from spamcop or do you forward them to yet another account?

The headers you provided do not indicate this message went through spamcop at all.

[Wazoo]

Wazoo, this is NOT a reporting issue but a mail handling issue.  The OP believes it HAS to be spamcop causing the problems.

I could agree with you, but I'm looking at the first post that allegedly contained an e-mail from this user's hosting ISP;

Its very odd-- if I have your email go from a qmail-1.03

server to a sendmail (8.12.8) server, somehow spamcop doesn't like the headers.

No other servers have any problems. Thanks, Dan"

Which to me is defining an internal issue with that ISP's e-mail / server set-up. From this end, am I supposed to guess that the background to this response is that there are other servers running qmail-1.01 and / or sendmail (8.10.9) .. and traffic through this combo of servers makes it to / through the SpamCop parser just fine ...??? Again, maybe it's just me, but that's the way I interpreted that note.

The headers you provided do not indicate this message went through spamcop at all.

I'm still sticking with the "user's definition of [the body]" in that sample as being an issue, thinking that a whole bunch of data didn't make it from whatever he/she is talking about to the stuff that was displayed here. And again, I'll admit to running with assumptions and guesses at this point ....

For example, user states "Again, no subject indicated in these messages" ... yet I see a Subject: Line clear as a blue sky .... So either I can't guess at what those words meant or I go with that those words were describing something not actually included in the sample data pasted in ...????

[jrsherrard]

Sorry about my lack of clarity.

I am once again receiving emails with complete subject lines and have been since I first posted here yesterday. My original posting was an attempt to figure out if this was a known issue or not. I've gathered, from the responses, that it is not.

The email I quoted from my ISP simply expressed their mystification as to why their qmail 1.03 server sending to spamcop should respond (only occasionally) by mangling the subject lines.

The email I quoted in my second post yesterday was what the text of the subjectless email looked like - ie, the exact body text of the email.

My email is forwarded from my ISP to spamcop, and I use pop.spamcop.net to retrieve.

And the truth is, I don't know what the conflict was and neither does my ISP. Its just that it's now happened twice and it's puzzling.

I have emailed my ISP and will post any other relevant info he comes up with.

Thanks so much for your time.

Jean

[Wazoo]

Once again, if you want someone "here" to see what is happening, you need to provide the Tracking URL of one of the failed items. This is the only way to analyze what it is exactly that is being "seen" by the SpamCop parser. Anything else is just muddying the waters a bit more.

[jrsherrard]

Does the following provide any clues? I see that X-spam status recognizes there's a missing subject.

If this doesn't include it, where would I find the tracking URL?

From paul[at]dorpat.com Fri Oct 8 20:41:18 2004

Return-Path: <paul[at]dorpat.com>

Delivered-To: spamcop-net-sherrard[at]spamcop.net

Received: (qmail 12677 invoked from network); 9 Oct 2004 03:40:02 -0000

Received: from unknown (192.168.1.103)

by blade6.cesmail.net with QMQP; 9 Oct 2004 03:40:02 -0000

Received: from news.zipcon.net (209.221.136.9)

by mailgate2.cesmail.net with SMTP; 9 Oct 2004 03:40:01 -0000

Received: from zipcon.net (zipcon.net [209.221.136.5])

by news.zipcon.net (8.12.8/8.12.8) with SMTP id i993XGSC032038

for <sherrard[at]spamcop.net>; Fri, 8 Oct 2004 20:33:16 -0700

Date: Fri, 8 Oct 2004 20:33:16 -0700

From: paul[at]dorpat.com

Message-Id: <200410090333.i993XGSC032038[at]news.zipcon.net>

Received: (qmail 4990 invoked by uid 565); 8 Oct 2004 20:42:37 -0700

Delivered-To: sherrard[at]zipcon.net

X-spam-Checker-Version: SpamAssassin 3.0.0 (2004-09-13) on blade6

X-spam-Level: *

X-spam-Status: hits=1.7 tests=MISSING_SUBJECT,MSGID_FROM_MTA_HEADER,

NO_REAL_NAME version=3.0.0

X-SpamCop-Checked: 192.168.1.103 209.221.136.9 209.221.136.5

X-SpamCop-Disposition: Blocked bl.spamcop.net

X-SpamCop-Whitelisted: paul[at]dorpat.com

[Wazoo]

I deleted the "all quote but nothing else" post.

Your query as to "does this provide any clues" is not defined as to whether or not this is a "failed" parse item or not. And there is no immediate sign of your original concept of "stripped headers, dumped into the body" .. so I have no idea what I'm really supposed ot be looking at / for .. and again, not knowing what you've left out, added in, etc., etc.

Tracking URL is defined in the FAQ entry under GLOSSARY ....

[StevenUnderwood]

jrsherrard,

Wazoo is tryng to get you to submit one of these messed up messages to the parser so we can see the FULL message as your client interpreted it (full headers and body). When you parse it, you will see the tracking URL at the top of the parser. You can cancel the report after you have obtained the URL.

The spamcop system now stores the message source for a while, allowing us a chance to see exactly what you were submitting.

When you post the same thing in this forum, the forum software modifies it so we can not always tell what an email program will interpret as headers or body.

[jrsherrard]

Thanks again for your patience. I've only ever submitted spam to be parsed, not knowing that there were other purposes for tracking URLs.

All right then. Here's the tracking URL of one of the emails:

Here is your TRACKING URL - it may be saved for future reference:

http://www.spamcop.net/sc?id=z681571872z38...290553c7583407z

Any less muddy?

Jean

[Wazoo]

Here is your TRACKING URL - it may be saved for future reference:

http://www.spamcop.net/sc?id=z681571872z38...290553c7583407z

Any less muddy?

Gads ... let's start with what we now see. Your Tracking URL also allows us to see what was actually submitted at http://www.spamcop.net/sc?id=z681571872z38...;action=display

There are two sets of headers there, and based on the "normal" construction of an e-mail, the "problem" is that the parser engine is working on the headers that reference the e-mail from your ISP to SpamCop, thus resulting in your ISP as being the source of that e-mail .. which it is.

As to where this second header is coming from, I'm at a bit of a loss. What is happening, best as I can try to describe it, is that your spam submittal is being added in-line to another e-mail. You keep keying on the missing Subject: Line, but I'll also point out that there is no line that suggests what application was used to develop / process this "new" e-mail. I've re-read this whole Topic, but I can't yet tell exactly how this spam submittal gets generated and sent, so I can't yet figure out whether you are actually generating this second set of headers (thinking that if you did, the application of this mailer would be identified) or if this is the actual situation your ISP is tlaking about ... the action of going from a qmail-1.03

server to a sendmail (8.12.8) server is what is generating this second set of headers, which I can't even begin to try guess at why this would be happening. (Hopes are that perhaps another admin might chance across this and offer some insight?)

But, the bottom line is that this isn't a problem with the SpamCop parsing and reporting tools. They are working just fine. What needs to be discovered is exactly where and how the additional set of headers gets prepended to the actual spam submittal.

Am now wondering if the posting of a "successful" parse might reveal anything or only reinforce that it only happens when the path taken by the e-mail is just so ...???

At this point, as it is now seen as a Reporting issue, this Topic is being moved over into the Help Forum.

[StevenUnderwood]

Wazoo (and jrsherrard please correct any wrong statements here):

The way I am reading this thread, I am back at the original problem being that when the message is received in the spamcop email inbox, the message is messed up where the header lines are seperated so that some (including the subject) are being interpreted (correctly, if that parse is correct) as part of the body of the message. The top part of the headers does not contain a Subject field, so the webmail application wouldnot have anything to place there.

I am still having trouble trying to determine exactly how the message travels and seeing 2

From Lilloway<at>aol.com Fri Oct  8 19:10:55 2004

From Lilloway<at>aol.com Fri Oct 08 22:41:46 2004

with different time stamps only confuses me more. My guess as to the path of this message is:

Received: (qmail 12447 invoked from network); 8 Oct 2004 22:39:11 -0000

Received: from unknown (192.168.1.103)  by blade3.cesmail.net with QMQP; 8 Oct 2004 22:39:11 -0000

Received: from news.zipcon.net (209.221.136.9)  by mailgate2.cesmail.net with SMTP; 8 Oct 2004 22:39:11 -0000

Received: from zipcon.net (zipcon.net [209.221.136.5]) by news.zipcon.net (8.12.8/8.12.8) with SMTP id i98MWPSC018641 for <x>; Fri, 8 Oct 2004 15:32:25 -0700

Received: (qmail 22254 invoked from network); 8 Oct 2004 15:41:45 -0700

Received: from unknown (HELO imo-m23.mx.aol.com) (64.12.137.4)  by mail.zipcon.net with SMTP; 8 Oct 2004 15:41:45 -0700

Received: from Lilloway<at>aol.com by imo-m23.mx.aol.com (mail_out_v37_r3.7.) id 4.59.180360a4 (4529)  for <x>; Fri, 8 Oct 2004 18:38:59 -0400 (EDT)

And this would be the first header in the correct location, so I would assume it is the handler just before this header (perhaps the local spamassassin configuration: sim.zipcon.net) causing the problem.

Received: from zipcon.net (zipcon.net [209.221.136.5]) by news.zipcon.net (8.12.8/8.12.8) with SMTP id i98MWPSC018641 for <x>; Fri, 8 Oct 2004 15:32:25 -0700

Again, this is only a guess.

Is this an automatic forwarding to your spamcop account going on here? Is it possible to "forward and save a copy" with your ISP? Most cannot do this but some can. If so, you can compare the same message before and after the forwarding. Of course, this would need to be done when the problem is happening.

Do you have another account you could forward to either instead of or in addition to your spamcop account. See if you can eliminate the spamcop part of the equation (again while the problem is happening). You could even use a Yahoo type account for a test. Anything where you can view the entire headers of the message.

Good luck and keep us informed. And again, correct my assumptions if I am wrong.

[Wazoo]

Correct you? Assumptions wrong? If you could only see the scribbled notes I had going trying to sort it out. That's why I just focused on a couple of lines.

If you llok close, the top set of headers covers the transfer of this e-mail from zipcon.net to blade3.cesmail.net (and then the account) .. and this "flow" started at Date: Fri, 8 Oct 2004 15:32:25 -0700 (= 22:32:25 GMT)

The next set of headers shows the flow of this e-mail from Lilloway <at> aol.com to mail.zipcon.net (and then an account there?) .. and this all started at Date: Fri, 8 Oct 2004 18:38:59 -0400 (= 22:38:59 GMT)

Obviously, at least one of the servers involved is not in sync with real time.

The real stumbling from this end is trying to guess what happens at the line;

Received: (qmail 22254 invoked from network); 8 Oct 2004 15:41:45 -0700

in the bottom set of headers. Is the e-mail dropped into the user's InBox, is it forwarded back into the network to go back immediately, or something else. And not knowing where that approximately minus 6 minutes came from doesn't clear anything up.

And of course, that the last server identified in the bottom set of headers is mail.zipcon.net but the "first" machine to "handle" this e-mail in the top set of headers is news.zipcon.net ....??? same "network" but I don't "see" that handoff, and can't tell from here if it's automatic or the result of user actions .... ????

I'm thinking that the top most set of headers was generated "internally" while moving around that network .. but admitting that I'm only guessing. I'm still reeling a bit over the last one of these that Ellen stepped in and "fixed" somehow ..???

[jrsherrard]

Thanks so much to you both for spending so much time on this. I'm forwarding your analyses to my ISP. When I have a response, I'll post it.

Best--Jean

[Wazoo]

Believe me, we'd like to know what the actual problem is. I hate having to guess at things like this, darn it.