Apple Mail reporting setup for "no-body" spam.

[sbimos]

Using:

Mac OS X 10.5.6, Mail 3.5

[at]mac.com and [at]gmail.com email accounts

Spamcop reporting service

Typical Procedure:

from Apple's email client "Mail", I choose "Forward As Attachment" and send it to my unique Spamcop reporting address. *Normally this works fine*

Problem/Question (concise):

How do I add text to "no-body" spam if I report it using the "Forward As Attachment" command?

Problem/Question (detailed):

Recently I have been receiving poorly coded spam, and the body of the email appears empty. When I look into the headers, I can see the typical spam 'phrases' and 'websites'. Spamcop processes the submission and says "No body text provided, check format of submission. spam must have body text.". I understand this error, and the Spamcop suggestion to fix this is to "...add some text like 'No body included...'" (see http://forum.spamcop.net/forums/index.php?showtopic=122). However, I cannot figure out how I can still use my typical procedure of "Forward As Attachment" (the method that Spamcop accepts) if I need to add text. Suggestions? I am trying to keep the process as streamlined/fast as possible.

Sample "no-body" spam processing error:

http://www.spamcop.net/sc?id=z2690512304zd...99586638847952z

Laborious Workaround:

Currently, I have to reveal all headers, copy all, login to Spamcop, paste a report, add text, submit.

[rconner]

Do you have a Tracking URL from one of these unreportable spams that you could share here?

-- rick

[sbimos]

Do you have a Tracking URL from one of these unreportable spams that you could share here?

original post edited.

[Wazoo]

Recently I have been receiving poorly coded spam, and the body of the email appears empty. When I look into the headers, I can see the typical spam 'phrases' and 'websites'.

A Tracking URL just might make that sentence somewhat clear. As is, I really can't make much sense of the structure and content you're trying to describe. An e-mail header doesn't contain 'phrases and websites' ...???

However, I cannot figure out how I can still use my typical procedure of "Forward As Attachment" (the method that Spamcop accepts) if I need to add text. Suggestions? I am trying to keep the process as streamlined/fast as possible.

Laborious Workaround:

Currently, I have to reveal all headers, copy all, login to Spamcop, paste a report, add text, submit.

You have waded into a scenario that is not limited to your compter/OS/application choice. The act of 'forwarding an e-mail' as an attachment simply does not allow for the addition of content to that e-mail. There some examples available within this Forum of scripts others have developed to handle the Reporting process somewhat 'outside' of the AppleMail app itself, however, not sure that an actual "no body" e-mail has been taken into account.

But again, I'd rather see a real example of just what you're trying to play with. The scenario of 'broken' e-mail is certainly not a rarity, but .... noting that some have been figured out, some have been a lost cause. Not being able to 'see' what you're seeing makes it hard to discuss.

[rconner]

original post edited.

Thanks. Looks as though the message is getting clobbered. Might be by Apple Mail but I am not sure. After the last "Received" line, there appear to be no more line breaks (in particular, the two successive line breaks between header and body are missing, which makes SpamCop think there's no body).

I tried submitting one of my own spams in this way (I've never done it before), but it came through OK.

I would almost conjecture that the original spam had problems, but if you could report it by copy-paste without making any changes to it then that would probably not be the case.

I have successfully used the AppleScript located here on the forum, contributed by a fellow member, to report multiple spams. I think it uses a different approach, might be worth a try.

-- rick

[Wazoo]

Sample "no-body" spam processing error:http://www.spamcop.net/sc?id=z2690512304zd...99586638847952z

Lots of strangeness going on there. The first Received: line doesn't really make a lot of sense .. though one might guess that there's a server or three being used, possible spam filtering, load-balancing, ??? However, the real 'problem' is that there isn't a handoff showing just how the e-mail got from smtpin124.mac.com to from smtpin124-bge351000 .. maybe it's the same server???? (perhaps something as trivial as a computer with two network cards, one for WAN, the other for LAN???) Not necessarily critical, as it's a non-routable (so assumed to be internal) IP Address ... the parser ignores it.

The second Received: line starts the 'interesting' part. The later Date-warning: line states that this server didn't like/see existing data, so it added stuff .... both a new Date: line and a Message-ID: line. However, I can't see from the displayed data any 'good reason' that the timestamp in that second Received: line got busted, dropping the 12 Mar 2009 01:19:04 -0700 (PDT) to a new line (with no leading whitespace) ....

That strangeness is all the more 'interesting' as this is exactly where the header construct goes bad in the third Received: line .... the timestamp is 'broken' .. and from then on one sees the lack of 'newline' characters/actions ... and most definitely the missing blank line between the headers and the single line of text that wuld have made up the body.

To my eyes, I would first of all ensure that there isn't some kind of corruption found on your own hard drive. After that check comes back good, then I suggest sending this to someone on the networking side of mac.com and see of they can come up with a possibly good explanation as to why their systems would accept and pass on garbage like this. Worst case, are all of your "bad" e-mails crossing this same server?

A "fixed" version of this e-mail is parsed at http://www.spamcop.net/sc?id=z2691355026z1...03459b9664972ez with basically the same results for the source ..... noting that the spamvertisd URL report would be going to a "don't care" ISP anyway ....

[sbimos]

I would almost conjecture that the original spam had problems, but if you could report it by copy-paste without making any changes to it then that would probably not be the case.

I have successfully used the AppleScript located here on the forum, contributed by a fellow member, to report multiple spams. I think it uses a different approach, might be worth a try.

-I was trying to avoid the copy paste route, but I will try adding breaks the next time I get one. I have already deleted to most recent "no-body" spam.

-Actually, I DID have to make changes (add text) when I copied/pasted to report.

-Fantastic. That link/scri_pt should be added into the FAQ. I looked for exactly that and had only seen one for Eudora (if I recall correctly).

-The scri_pt does what it's supposed to do. I guess now I will work on modifying the scri_pt to add some sort of innocuous text to every submittal to address the "no-body" error.

-I'm not "Mr. scri_pt", but I added some more info to that post to help other users install and run the scri_pt.

That strangeness is all the more 'interesting' as this is exactly where the header construct goes bad in the third Received: line .... the timestamp is 'broken' .. and from then on one sees the lack of 'newline' characters/actions ... and most definitely the missing blank line between the headers and the single line of text that wuld have made up the body.

To my eyes, I would first of all ensure that there isn't some kind of corruption found on your own hard drive. After that check comes back good, then I suggest sending this to someone on the networking side of mac.com and see of they can come up with a possibly good explanation as to why their systems would accept and pass on garbage like this. Worst case, are all of your "bad" e-mails crossing this same server?

-You're already over my head with your analytics...

-No corruption. I get the EXACT same sort of thing about once a week, always same issue, looks like it's from the same sender. None of my other email ever has this issue. I disk-utilitied-permission/disk-repaired just to be thorough.

-I don't feel comfortable reporting the details of the issue to mac.com as I don't understand it myself. However, I AM reporting all spam received via their servers to the mac.com reporting address, so hopefully they have picked up on the issue themselves.

[rconner]

-I was trying to avoid the copy paste route, but I will try adding breaks the next time I get one. (...) -Actually, I DID have to make changes (add text) when I copied/pasted to report

Ummm...I'd not add any line breaks or make any other substantive alteration -- this may run foul of the strict SpamCop rule that you not alter the content of the mail in order to make SpamCop find things it otherwise would not. Yes, it appears that we are allowed to add a line to a blank e-mail, but this seems to be considered a permissible (or tolerated) bending of the rule (and doesn't actually make the results of the parse any different).

If you did this sort of surgery to the spam when you pasted it into the form (as opposed to just hitting return twice and typing in a little message at the bottom), this would certainly account for differences in parser results. The answer to the problem, however, is not to keep altering the spam but to get the mail service (mac.com?) to fix it as Wazoo suggests.

-Fantastic. That link/scri_pt should be added into the FAQ. I looked for exactly that and had only seen one for Eudora (if I recall correctly).

-The scri_pt does what it's supposed to do. I guess now I will work on modifying the scri_pt to add some sort of innocuous text to every submittal to address the "no-body" error.

-I'm not "Mr. scri_pt", but I added some more info to that post to help other users install and run the scri_pt.

-You're already over my head with your analytics...

I played around with the scri_pt myself, to add a means to let me select which of my several e-mail addresses to send from. Seems to be a pretty simple and robust scri_pt, but you are right that it could use some installation notes. I would be interested to know, however, if this scri_pt has solved your original problem.

-I don't feel comfortable reporting the details of the issue to mac.com as I don't understand it myself. However, I AM reporting all spam received via their servers to the mac.com reporting address, so hopefully they have picked up on the issue themselves.

One thing you might do for us is to examine the mail as it sits in your Apple Mail spool. Do you see the same things at the end of the header that we see in the tracking URL you provided? If you do not, then you may have a problem with Apple Mail. If you do, then you may have a problem with mac.com. You could report this problem to them and actually give them the tracking link you gave us so they can see where the stuff is difficult to parse.

-- rick

[rconner]

Lots of strangeness going on there. The first Received: line doesn't really make a lot of sense .. though one might guess that there's a server or three being used, possible spam filtering, load-balancing, ??? However, the real 'problem' is that there isn't a handoff showing just how the e-mail got from smtpin124.mac.com to from smtpin124-bge351000 .. maybe it's the same server???? (perhaps something as trivial as a computer with two network cards, one for WAN, the other for LAN???) Not necessarily critical, as it's a non-routable (so assumed to be internal) IP Address ... the parser ignores it.

Looks like an internal relay inside mac.com -- verizon used to do things this way to my mail, putting unrouteable IPs in the headers and breaking the hostname chain, but they have since gotten back on the stick (mostly). I gather that sbimos made good use of the Mail Host Configuration features, as SpamCop did not have any trouble with this info.

That strangeness is all the more 'interesting' as this is exactly where the header construct goes bad in the third Received: line ....

Coincidentally or not, this third Received line (from rhzogi) is where the forgery begins, also where the bus left the highway and line breaks started getting removed from the right locations and put in the wrong ones. It is possible, of course, that the spam might have been composed this way (i.e, by a moron). I asked sbimos above to see whether the spam header on his computer looks like the one in this tracker.

-- rick

[sbimos]

If you did this sort of surgery to the spam when you pasted it into the form (as opposed to just hitting return twice and typing in a little message at the bottom), this would certainly account for differences in parser results. The answer to the problem, however, is not to keep altering the spam but to get the mail service (mac.com?) to fix it as Wazoo suggests.

I played around with the scri_pt myself, to add a means to let me select which of my several e-mail addresses to send from. Seems to be a pretty simple and robust scri_pt, but you are right that it could use some installation notes. I would be interested to know, however, if this scri_pt has solved your original problem.

One thing you might do for us is to examine the mail as it sits in your Apple Mail spool. Do you see the same things at the end of the header that we see in the tracking URL you provided? If you do not, then you may have a problem with Apple Mail. If you do, then you may have a problem with mac.com. You could report this problem to them and actually give them the tracking link you gave us so they can see where the stuff is difficult to parse.

-I added something that would not run afoul of the Spamcop rules, but it was so late lastnight, I don't remember what it was.

-Regarding [at]mac.com fixing things, like I said, I'm not comfortable giving them feedback on the technical details, but I still report the spam "fwd as attach..." and add a note about the spam strangeness in layman's terms. I honestly think that's the best way to get them to deal with it anyway. Usually, if I don't include a note, the report would be bounced.

-Oooh, can you post your mod scri_pt?

-Regarding examining the mail, you lost me at "spool"...

-Regarding does this scri_pt eliminate the "no-body" issue, I already deleted the original spam, so I will try with the next one.

Coincidentally or not, this third Received line (from rhzogi) is where the forgery begins, also where the bus left the highway and line breaks started getting removed from the right locations and put in the wrong ones. It is possible, of course, that the spam might have been composed this way (i.e, by a moron). I asked sbimos above to see whether the spam header on his computer looks like the one in this tracker.

-You lost me at "Mail Host Configuration features"...

Moderator Edit: way too much stuff being "quoted" .. edited out bunches of stuff to conserve both vertical and disk space.

[rconner]

-Oooh, can you post your mod scri_pt?

Gotta get it off my old PowerBook, maybe sometime over the weekend.

-Regarding examining the mail, you lost me at "spool"...

Sorry for the jargon-slinging, let me try to rephrase the story so far.

We know that the e-mail you posted in the tracking link is seriously broken -- the header is not formatted in accordance with proper SMTP (e-mail standard) practice, which is why SpamCop can't fully decipher it. If the spam arrived at mac.com in this condition, this means that it was the spammer who hosed it up (not unheard of, these guys frequently get things wrong). If it did NOT, then something done by mac.com's servers, or by your computer, was responsible for the damage.

What we (or I at any rate) would like to know is what the spam looked like when it hit your computer. To do this, you would select the message in Apple Mail, then do View >> Message >> Raw Source (command-option-U) to see the original mail packet (you probably do this already in your copy-paste reporting on the SC web form). If you saw the same damage there as was submitted to SpamCop, then we could perhaps rule out your software.

If your software is not to blame, then it might be mac.com, but at this point I am more inclined to blame the spammer since all of the damage occurs ONLY in portions that the spammer himself had control over (i.e., the forged portion of the header down through the rest of the mail).

What do I mean by "that the spammer had control over?" Specifically, the line beginning "Received: from rhzogi", and the rest of the message thereafter, are entirely the fictional creation of the spammer -- routine header-line forgery of the sort that has gone on for a decade or more (this link to my website may be of help in understanding forged header lines). This spammer, however -- if he is indeed the source of the damage to the header -- did not know how to construct a parseable header line, nor to include a blank line between header and body (as required by SMTP). As a result, his mail is incomprehensible to strict SMTP parsing, and probably to most mail clients as well.

-You lost me at "Mail Host Configuration features"...

I was assuming that you had put your e-mail addresses through the SpamCop Mail Hosts Configuration process when you signed up for a reporting account. This process helps SpamCop understand how your mail is normally routed, so that it can more accurately determine where it comes from. If you didn't do this, it might be a good idea to do it now. See http://www.spamcop.net/fom-serve/cache/397.html for the details.

-- rick

[sbimos]

Odd, Spamcop forum didn't notify me that anyone had responded yet... I'll have to catch up on those later.

Update:

I received a fresh no-body spammy-spam today. I ran the original scri_pt and it did not fix the problem. I ran a modified version of the scri_pt, and it worked just as expected:

http://www.spamcop.net/sc?id=z2695227880zc...f0773d223c8b30z

I think I am satisfied with the result now. I will probably make two copies, one each for [at]mac and [at]gmail, both will also go to Spamcop, and to that FTC.gov address. scri_pt as follows:

-added lots of instructions

-switched to BCC

-added the 'add two breaks and note to end of the file'

(* SendToSpamCop V2.0

written by S.J.L. v/d Velden

The code below is open source and you may freely edit and redistribute it. Though I would appreciate it if you gave credits to the original author (me).

Modified by Sbimos and Legioss to allow reporting of "no-body" spam, reporting to multiple services, reporting via bcc, and use of Address Book groups.

Assistance from:

StefanK at Macscriptor.net

RConner at Spamcop.net

Wazoo at Spamcop.net

--------------------------------------------------------

You have to customize the recipient variable to make this scri_pt

work with your SpamCop-account and, or, other reporting services.

Below you see the line that begins with "set SCaccount..."

Replace the text YOURACCOUNT with the intended recipient address or addresses.

Use the personalized reporting address you received from SpamCop.net to forward your spam to, or the reporting address of other services you may use.

If you are using multiple reporting services, seperate the email addresses with a comma and a space.

If you have an Address Book group, simply replace with the group name.

*)

--

set SCaccount to "YOURACCOUNT"

(* Below is the rest of the sourcecode of the scri_pt.

Please be very carefull editing this code *)

-- Create a SpamCop folder on the desktop

set theOutputFolderPath to path to desktop folder

set theNewFolderName to "SpamCop"

tell application "Finder"

if (exists folder (theOutputFolderPath & theNewFolderName as string)) = false then

make new folder at desktop with properties {name:theNewFolderName}

end if

end tell

-- Create a new message in mail bcc addressed to the user's SpamCop account, and, or, other reporting services. Read the source from the selected messages in mail and save it as SpamCop readable file in the newly created SpamCop folder. Then attach each file to the new message. Two line breaks and note "(This line added to ensure proper spam reporting.)"appended to the end of each file.

tell application "Mail"

set theMessages to the selection

set counter to 1

set theMessage to (make new outgoing message with properties {visible:true, subject:"report spam", content:" "})

repeat with thisMessage in theMessages

set sourceFile to ((theOutputFolderPath & theNewFolderName as string) & ":ml" & counter & ".src")

set thisSource to the source of thisMessage as string

set f to open for access sourceFile with write permission

set eof of f to 0

write thisSource & return & return & "(This line added to ensure proper spam reporting.)" to f

close access f

tell "Finder"

set theAttachment to sourceFile as alias

end tell

tell the theMessage

tell content

make new attachment with properties {file name:theAttachment} at before the first character

end tell

end tell

set counter to counter + 1

end repeat

tell theMessage

make new to recipient at end of bcc recipients with properties {address:SCaccount}

end tell

end tell

[rconner]

I received a fresh no-body spammy-spam today. I ran the original scri_pt and it did not fix the problem. I ran a modified version of the scri_pt, and it worked just as expected:

The tracker you posted shows just the same kind of damage as the one from the other day. So, the new scri_pt did not change anything (or are you saying it did?). The good news is that this probably lets your computer & software off the hook.

-- rick

[Wazoo]

Odd, Spamcop forum didn't notify me that anyone had responded yet...

A few of the previous times this has come up ....

http://forum.spamcop.net/forums/index.php?...ost&p=67487

http://forum.spamcop.net/forums/index.php?...ost&p=68775

http://forum.spamcop.net/forums/index.php?...ost&p=66196

[sbimos]

The tracker you posted shows just the same kind of damage as the one from the other day. So, the new scri_pt did not change anything (or are you saying it did?). The good news is that this probably lets your computer & software off the hook.

WTF? I know it worked, because that was the whole reason I was satisfied! I tried it again, but the spam is too old. I'll wait for the next one and try again. FYI, a spam-IN-body (normal spam) will still work with the scri_pt... if anybody didn't figure that out already.

Idea:

A colleague suggested that instead of adding two text breaks and my "(This line added to ensure proper spam reporting.)", that I set the scri_pt to add two text breaks and then DUPLICATE the original contents of the spam. That way I haven't technically added anything other than the text breaks.

How does that sound?

A few of the previous times this has come up ....

I set them to "immediate", so we'll see how that works. I didn't catch what they were before I changed them.

What we (or I at any rate) would like to know is what the spam looked like when it hit your computer.

Finally checked it. They are exactly the same.

I was assuming that you had put your e-mail addresses through the SpamCop Mail Hosts Configuration

I joined YEARS ago, so I don't think I did. Crud, it looks like I need help with that too:

http://forum.spamcop.net/forums/index.php?...f=7&t=10170

[sbimos]

The tracker you posted shows just the same kind of damage as the one from the other day. So, the new scri_pt did not change anything (or are you saying it did?).

Aha! Here's a new one. Got another "no-body" this morning. Processing fine by spamcop. I screenshotted it just in case, to prove I'm not loony.

http://www.spamcop.net/sc?id=z2705043662zc...9dc5c6d8586648z

Ohhh... As soon as I sent the spamcop report, reloaded the above link, it added this line: "Reports regarding this spam have already been sent:" and at first glance it LOOKS like it didn't process.

[rconner]

SpamCop was able to read enough of the header to get to the source, and was able to generate reports for it. It could not get at the URLs, since they were buried in the mangled part of the message. So, yes, SpamCop did file reports, but this does not mean that the message isn't damaged.

If the message hadn't been mangled, there's a possibility that SpamCop would have tracked down the website URL and offered to report it for you.

There's more wrong with these headers than simply the lack of a blank line between header and body. We might be able to read this mess and guess what to fix, but it would be a GUESS -- not the sort of thing we are supposed to be doing with mail submitted to SpamCop.

-- rick

[sbimos]

There's more wrong with these headers than simply the lack of a blank line between header and body. We might be able to read this mess and guess what to fix, but it would be a GUESS -- not the sort of thing we are supposed to be doing with mail submitted to SpamCop.

Yeah, I noticed it was skipping the URLs too. That's why I was wondering if setting the scri_pt to duplicate all the info might be a better answer. It sounds like my current scri_pt, doing minimal changes is more acceptable...

P.S. I have now set up my mailhosts.

[Wazoo]

Not stated:

Change to 'immediate' and notification receipt results.

Whether all the 'bad' e-mail is coming across the same smtpin124.mac.com or not.

Definitely noticed: the continued lack of editing down the quoted material in follow-on posts, perhaps not noted because I continue to edit your posts to remove the unnecessary data which consumes vertical space on the display, database space on the hard drive of this system.

[rconner]

That's why I was wondering if setting the scri_pt to duplicate all the info might be a better answer.

That's your decision, but here's the rub -- the rule is pretty explicit that you may not "...make any material changes to spam before submitting or parsing which may cause SpamCop to find a link, address, or URL it normally would not, by design, find" (my emphasis). Obviously, SpamCop did not find the link before, and if you change the spam in such a manner that it does find it, you are making an unallowed "material change." You said that a colleague advised you to change the spam; unless this colleague is a paid employee of SpamCop speaking as such, I might say that his advice isn't very relevant.

There's a very good reason for this rule. If people are allowed to make tiny changes now, then in a few months they may be allowed to make bigger changes, and bigger changes still, until at last they are submitting messages that are radically different from those they received. A few people here know how to "fix" broken messages like this one, but most do not, and their changes may create big problems. SpamCop may make wrongful reports based on these altered messages, and word will get around that SpamCop reports are not trustworthy and are in fact arbitrary and vindictive.

As you can see, there are some "official" exceptions to the rule, but in most cases they do not actually break the rule (i.e., they don't cause things to be found that would not normally be found). Adding a blank line followed by some neutral text is an example of one of these exceptions; if you put URLs etc. into this neutral text you would then be running right into The Rule.

You do not have to depend upon SpamCop to report this website; you can do it yourself if you care to try. See this Wiki link for more information on this.

Also, you should be aware that most hardcore spammers' websites are pretty impervious to isolated complaints, so even if you go to the trouble to send a report it is probably not going to accomplish very much. You might have more impact doing something like submitting your spam to KnujOn.

-- rick

[sbimos]

-[at]Wazoo: Seriously, I was TRYING on the quotes, sorry.

-[at]Rconner: I try to keep my reporting ultra-simple, so I will avoid personal reports.

-[at]Rconner: I hadn't heard of Knujon before, Thanks.

-[at]Rconner: So should I change my scri_pt from "(This line added to ensure proper spam reporting.)" to something even more minimal like a " " or a "."?

[rconner]

So should I change my scri_pt from "(This line added to ensure proper spam reporting.)" to something even more minimal like a " " or a "."?

Not as far as I know. The statement you are using contains no links for SpamCop to "find"

-- rick

[sbimos]

I'm back. The old computer died, and I completely forgot to get my custom scri_pt off the backup drive, so I am starting over (back a few steps).

I reinstalled the scri_pt above, and now I am getting an error from Spamcop that I thought was fixed in the previous incarnation of the scri_pt:

See sample report below:

http://www.spamcop.net/sc?id=z5014485383z5...894e78d3ead746z

Ok, what do I need to do?

Thanks

[Farelf]

Missing the full "Received:" line as shown in example at http://www.spamcop.net/fom-serve/cache/17.html (no IP address for received from).

Perhaps others can advise on scripting to capture that.