Hello,

I am getting spam with multiple blank lines between the header and the body...

(i) is Quick Reporting reporting or dropping spams of this nature?

(ii) if using the 'full reporting', can I legitimately edit the spam to have just one blank line between the headers and the body, and the parser will process the header at least? (e.g., http://mailsc.spamcop.net/mcgi?action=gett...rtid=3419147922)

(iii) how should I edit the spam to get the parser to see the body as well? (as Thunderbird sees it just fine, if I choose to open it)

(iv) If this sort of editing is "OK" (ish), do I need to add a note to that effect? (much like adding a "[no body]" note as a body for header-only spams?)

Cheers:
skiwi

What is the problem? More than one blank line is nothing more than additional (vertical) whitespace .. ignored during the parse ....

Your link is basically useless. The 'mailsc' link limits access to members with paid accounts. The 'report ID' part is an item addressed within the SpamCop FAQ here. the point being that a Report ID is of value only to 'you' ....

In addition, the Forum FAQ clearly states that multiple accounts here are not allowed. Please state which account you'd prefer to have deleted. Unfortunately, you have chosen to post under both accounts, so there's a bunch of manual updates needed to clean things up, a lot of work that should not be needed.

[On edit - tacking this on to Wazoo's post so as not to potentially create more clean-up work.

I suggest the body of the spam in question might have looked like this one:
http://marc.info/?l=alice&m=121971067605207&q=raw
... in which case, there are certainly no problems for the parser.]

thanks for the reply...

I was not at home - so thought I might use the forums... feel free to delete whatever you see as appropiate here in the forums

I will return to the better utility / less 'fluff' of the newsgroups I lurk in... post the spam in .spam in its entirety, no messing with this or that link, and then refer to it in a 'regular' newsgroup post

Just FYI (and I recognise that this is a little esoteric as you can not see the spam itself):

- as received
* the spam can be opened in an email client 'properly'
* the spam will not pass at all in the SpamCop parser

- removing the multiple blank lines between the header and the body allows the
header to be parsed, but not the body...

cheers!

That does not compute with the way the parser works. I can introduce multiple blank lines into a spam and still have it parse the headers and body correctly. You will likely still need to provide tracking URL's over in the newsgroups to show what you are seeing.

http://www.spamcop.net/sc?id=z2189498572zf...97a4d0c2ba8f97z

http://www.spamcop.net/sc?id=z2189500776za...aee3efff981bcez

purely FYI, as you seemed interested - here is an example that will parses (or not) exactly as I describe above

before it is mentioned by the 'forum custodians', no link/URL posted as I did not submit it in any way, shape or form...

:-)


-------------------
Return-Path: <storemu[at]bar-plate.com>
Delivered-To: x
Received: (qmail 13804 invoked from network); 26 Aug 2008 18:46:59 -0000
X-spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on blade5
X-spam-Level: *******************************
X-spam-Status: hits=31.7 tests=HELO_DYNAMIC_DHCP,INVALID_MSGID,MISSING_DATE,
MISSING_HB_SEP,MISSING_HEADERS,MISSING_SUBJECT,MSGID_OUTLOOK_INVALID,
MSGID_SPAM_LETTERS,RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_DYNAMIC,
URIBL_AB_SURBL,URIBL_BLACK,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_RHS_DOB,
URIBL_SC_SURBL,URIBL_WS_SURBL version=3.2.4
Received: from unknown (192.168.1.108)
by blade5.cesmail.net with QMQP; 26 Aug 2008 18:46:59 -0000
Received: from dsl88-246-47407.ttnet.net.tr (88.246.185.47)
by mx71.cesmail.net with SMTP; 26 Aug 2008 18:46:58 -0000
Message-ID: <6360______________________5e04[at]bar-plate.com>
From: "=?windows-1251?B?QWxpc3RhaXIgQXJub2xk?=" <storemu[at]bar-plate.com>
To: <skiwi[at]spamcop.net>
Subject: =?windows-1251?B?U29sdXRpb24gZm9yIHlvdXIgc2V4dWFsIGxpZmU=?=
Date: Tue, 26 Aug 3609 21:47:02 +0300
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary=----=_NextPart_000_0023_C5_FD803DA7.C6405FF8
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.2180
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180
This is a multi-part message in MIME format.
------=_NextPart_000_0023_C5_FD803DA7.C6405FF8
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-SpamCop-Checked:
X-SpamCop-Disposition: Blocked SpamAssassin=31







The ultimate convenience store in drugs, brought to=20
you in just one click!
=20
Select from thousands of prescr. drugs to be=20
delivered right to your doorstep.
=20
- V & C, Tram, Som all available
=20
- Express delivery
=20
- Secure checkout via credit card
=20
- No limit to quantity ordered
=20
- NO DOCTOR'S VISITS - all orders are filled=20
inhouse and shipped out straight to you
=20
Don't pay a single cent more than you have to for=20
the meds you need, today.
=20
Click here
------=_NextPart_000_0023_C5_FD803DA7.C6405FF8
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; iso-8859-1">
<META content=3D"MSHTML 6.00.2900.2180" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>The ultimate convenience store in drugs, b=
rought to=20
you in just one click!</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Select from thousands of prescr. drugs to =
be=20
delivered right to your doorstep.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>- V & C, Tram, Som all available</FONT=
></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>- Express delivery</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>- Secure checkout via credit card</FONT></=
DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>- No limit to quantity ordered</FONT></DIV=
>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>- NO DOCTOR'S VISITS - all orders are fill=
ed=20
inhouse and shipped out straight to you</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Don't pay a single cent more than you have=
to for=20
the meds you need, today.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2><A=20
href=3D"http://duringfell.com">Click here</A></FONT></DIV></BODY></HTML>
------=_NextPart_000_0023_C5_FD803DA7.C6405FF8--

The above area is the appropriate section for the blank line as that is where the boundry is. The fact that removing the blank lines below that allows you to get a correct parse (even though I don't yet understand why it gives you anything different) is really not part of the problem (or a real fix).


The fact that the spamcop headers are at the top of those blank lines seems to indicate that is how spamcop received it.

hence my confusion! :-)

I will FREELY admit I just rely on the SpamCop parser to do its thing... :-)

I do get tired of discussions without any real data. I submitted this peace of spam using the web page to cut and past in the spam. Then between these two lines

I inserted several line feeds before submitting it to the parser. As you can see this parses just fine. Being "modified" I unchecked all the blocks and had the parser send me a copy of the report. This makes the URL work and not report the spam.

{edit} On second look I see that the parser no only identified the source of the spam but also identified the "Administrator of network hosting website referenced in spam", which is below the multiple blank lines. {/edit}

Perhaps we could see other examples.

I think that something is scrambling up your headers. These lines:


Should be in this order:



So, my question is....what is scrambling your headers? I'm going to guess that it's whatever software and/or method you're using to obtain the raw message format. I too have a SpamCop email account and my headers never arrive jumbled up like that. I'm pretty sure that's your problem....and that it's on your computer.

DT

You can also just cancel the parse, saving the URL from the top of the parse. This is a bit safer in my estimation as when you submit, my understanding is that the spam is counted on the BL figures.

Could be. After doing the parse (to see how/if it worked) I got to the submit reports. When I canceled there I was left with a "Report Now." Guess I should have canceled then? (wouldn't expose an email address ether.)

I can go back to bed now, I've learned something today.

Thanks, that *is* the same body that is in the link I tacked on to Wazoo's post. There are two things fundamentally wrong with the spam you have posted

• the boundary string when it is declared in the header doesn't match the boundaries in the body (needs to be:
  boundary=------=_NextPart_000_0023_C5_FD803DA7.C6405FF8)
• The one critical space between the headers and the body is missing (needs to be:
  X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180
  
  This is a multi-part message in MIME format.)

See http://www.spamcop.net/sc?id=z2190761209z4affc6659dff0762ac90115a832030d5z with just those minimal changes made.
That one hasn't found the spamvertized link, could still be things wrong.

DavidT pointed out the several misplaced X-SpamCop lines. They don't prevent the parse in themselves but very likely do indicate the source of the mangling (though the mangling seen in this case almost defies belief). Not editing the spam in a spreadsheet are you? Nah, probably just the way the spammer made it. But then getting a successful parse?? Just doesn't add up. Unless you are pasing the headers and body seperately into the 2-part "outlook/eudora workaround form" boxes in the webform submission page. That would be rather naughty in this context (not giving us the information to work with).

Good luck going to the newsgroups - give them more information, like your email client(s), confirm the submission method(s) and tool(s) for failed and successful results, etc.The heavy hitters "over there" seem to prefer the tracking links pointing to the parsed result these days, just like here. Reason being you introduce irrelevant tools and processes when you copy and paste spam to a bulletin board or to a newsgroup (had to restore the offsets in the stuff you pasted here, just for starters). And it is all too easy to paste the wrong 'version', something other than what is producing the results being discussed. But go for it, they are better placed to discuss their own 'druthers.

Ah no, that would be the old iso-8859-1 7-bit/8-bit problem I guess, that
<A href=3D"http://duringfell.com">Click here</A> construction looking familiar - along with all the "=20"s. Dodgy spammer mass mailing ware being the cause IIUUC. And SC is not optimized for dealing with "spamvertized sites" anyway.

FWIW, duringfell.com, a HiChina domain, being NXD right now while they hunt for new hosts and nameservers.

Here is the Tracking URL for that spam.

http://www.spamcop.net/sc?id=z2187243858z5...6a8b55309b08ddz

I only see one blank line after the headers.

- Don D'Minion - SpamCop Admin -
.

Interesting ... and thanks for that link Don, no more fruitless conjecture.

And, to the O/P - the defects I mentioned in my previous are all that prevent the 'real' spam from parsing all the way through too. But nothing can be done about that, in terms of legitimate SC reporting. It is unfortunate that the defective code is, you say, 'correctly' rendered by (some) email client. But it doesn't matter - reports would be/did get sent to the spam source anyway (parsing of the headers for that part is not actually affected) - and that is the 'mission'. And as said, the spamvertized website is out of the picture anyway, however briefly.

And that message source doesn't display the same "headers/body" jumble that I pointed out earlier. We need for "Scuba" to come back and answer the question about how those mangled headers were obtained.

DT

This doesn't really help much in me trying to make a decision. You imply that if you find yourself somewhere else in the future, you'll do the same thing again ...???? Am I supposed to read into this that you've forgotten your original account data or try to come up with some other reason for generating another account?? (Which of course feeds into which account should be 'adjusted' ..)

That would fit in with the user description of "editing out all the extra blank lines" in the submittal. What would be needed would be a copy of one of the 'failed' attempts.


I just checked again (although not including spamcop.spam) but I don't see where any post has been made in any of the other newsgroups referencing this issue.

http://www.spamcop.net/sc?id=z2187237559z9...a3c3c540035c44z

I think that is the Tracking URL to the failed submission the OP is talking about. It appears to be the same spam as described above. Looks like the same msgid and time stamps as the other one.

There are multiple blank lines after the headers, but that isn't the problem.

The problem is that the headers are truncated. They're missing essential information, and so the parse balks because it knows the headers are incomplete.

- Don D'Minion - SpamCop Admin -
.

Thanks Don... THAT makes sense because looking at the parse, there is one LONG line there that would probably be fixed on any editing. It is strange that the "Display" function correctly displays this same header, however.

Actually, rather than truncation, a careful analysis demonstrates what I posted above -- that the preface to the first part of the "multipart" boundaries have been mangled into the headers, inserted before the "X-SpamCop" lines. These lines:


should appear in the body, not the headers of the raw message. They should appear *after* these lines (and a blank line):



So...it appears that something on the OP's end is mangling the raw source, commingling part of the body with the headers. But I pointed that out before and nobody really picked up on it. Please take another look and I think you'll find that I'm correct.

The OP hasn't bothered to come back, however, so this is all a bit of an exercise in futility....

DT

Yeah I did.Yep, he's taken it to the newgroups. Last log-in here was to make his post #7 above. Asking the same questions over there ... Mike Easter responding with his infinite patience and formidable insight/knowledge:

"There are a lot of things 'wrong with' the spamitem which you pasted into
.spam, and it is not possible for us to see the item as originally sent
before it was -1- munched on by SC's SpamAssassin filter -2- handled by
the submission process -3- changed by spurious linewraps from being pasted
into .spam" ...etc.

But it was a subtle acknowledgement ("DavidT pointed out the several misplaced X-SpamCop lines") and others have seemingly blown right past my analysis. Take another look, folks.


Right...I've just posted over there, too.

DT

A bit after my post that was meant to be a it of a jest over the fact that Mike E. basically said the same thing(s) you did. I'm thinking that Mike E. perhaps missed the <g>

Yes, I saw your post. What's needed in *either* venue is for the OP to reveal how he's obtaining the mangled headers. Also, I'm wondering about the submission methodology that produced this result:

http://www.spamcop.net/sc?id=z2187237559z9...;action=display

I'm guessing that the OP obtained the mangled/corrupted headers from an email client on his computer and submitted using the past-into-the-form method, but if that's not the case, we need to hear it from the OP.

DT

I've seen the 'invalid address' messages lately too... I simply quick report them, rather than run them through the full reporting.

FYI, this is what's causin the problem:



The parser is added the x-checked info after the content type. I assume that's due to a lame mail program the spammer used.

Well, no, not necessarily. We don't know for sure how the OP is submitting. In the first post, he asked whether or not Quick Reporting was working, but didn't say that's how these mangled messages are being submitted. He also mentioned using Thunderbird, so it's possible that he's getting the messed-up source from Thunderbird, but we can't know much for sure unless he graces us with his presence and answers some questions.

DT

David as you said this is all guess work, but I don't think Quick Reporting using Thunderbird is the issue. That is the process I use without problems. - Could be a bad install but...

Didn't I see somewhere in a header that the spam came through a SpamCop mail account? But others report from SC mail.

Oh will the OPer, having been told the same thing here and in the news groups and seems to have said 'think you' and gone.

...but that's not what I'm (speculating, due to an AWOL OP) is going on. I'm guessing that he's doing something himself that's mangling the headers, and then pasting them into the web-based reporting form.


Yep...and we never see the commingling of headers that was seen in the examples here. Hence, my theory that the problem might lie between the keyboard and the chair...


Something like that...whatever....a big waste of time, actually...on to something more productive.

DT

David: Never would be an overstatement. I have seen these (though very few and several years ago). They showed the same "long non-split" headers as thie original here did in the parser (but not in the display version). I assumed at the time that they were spammer mess ups because the headers affected are ones that would be provided by the sending system, not those added by the receiving server.

Are you referring to the TU supplied by Don?


Not sure I follow you. The headers misplaced in the example parse Don posted where not those from the sending server, but rather from the SpamCop email system...the X-SpamCop lines, which had body elements misplaced above them. I'm not understanding you, and I still think we're simply wasting time due to a lack of cooperation from the OP.

DT

The OP has given more detail on his submission process in the NGs - and clarified that the mangled headers-body were exactly how they were received. That would be the SECOND tracker Don provided. There may be things to learn but the NG is his preferred venue and that is where he has continued. If there is a conclusion over there it will be noted here.

Yes, the second TU:
http://www.spamcop.net/sc?id=z2187237559z9...a3c3c540035c44z

SpamCop's server is not looking for specific headers to place it's own headers, it is looking for that first blank line which indicates the end of the headers. I see no blank line before the spamcop headers where SpamCop "misplaced" it's headers.

Looking at the headers as the parser did (not as displayed in the "Display" view), the long line of headers starting with "Message-ID:" are all provided by the sending server, all the way to the "------=_NextPart" line. If the spamcop server did not detect any blank line until after that line, all those lines are detected as headers and SpamCop adds it's X-SpamCop-* lines at that point.

Even in the "Display" view, there is no blank line before the MIME boundary to indicate to the receiving server the the body is beginning and for SpamCop to add it's headers.

Now, I admit that this could be something the sending machine is doing or some problem with SpamCop's servers (under very specific conditions maybe) or some fluke in the transmission. In my experience it is very rare (way before the current set of servers was implemented).

That one long line is what is actually causing the unparsable result, however, as the parser does not see the individual headers in the line and as Don mentioned, the parser does not see all the headers it needs to see to determine it is a full set of headers. This explains why the display view parses fine (even without removing the multiple lines as done by the OP). It has fixed that one long line.

The one long line is the thing I have seen in the past.

Ah...thanks. Now I'm starting to get it. I'm left wondering why the "display" view would show us something other than the actual raw source of the message, in that the link that brings it up is labeled "View entire message." That view should, IMO, show us the raw source of the message, as submitted, with only the munging that the system performs to protect the identity of the reporter. If there's additional reformatting going on, that's news to me (and I rather doubt that it's covered anywhere in the FAQs).

DT

Wazoo asked that I give a SUM of what was noted over in the newsgroup in various responses, etc... (apologies by the way if taking it over there caused frustation and confusion, even though I mentioned that I was doing that...)

Here are some quotes with some snippage, including from my self, that I think might be pertinent to the original question - as well as confirming IMHO that I in no way mangled the submission somehow as has been suggested here and there...

scubak1w1 @ 08/31/08:


Mike Easter @ ### inline context with above, see newsgroup for that if you need this exactly:



SpamCop Admin, back channel @ 09/06/08:


Thanks for all your replies here as well, at least the ones that addressed the original question!

Wrong!! Wazoo said that feedback was needed in several places, starting with this Topic. What I was looking for was answers to a number of questions asked. As I stated in the newsgroup, I am still waiting for a response to my questioned scenario about your multiple acounts. Your offer of deleting both accountsz so you can then re-register yet another is not acceptable.