Outlook received header problem

[Farelf]

From http://forum.spamcop.net/forums/index.php?showtopic=10241 (O/P brantgurga) Added emphasis.

Based on the above data, results, and additional explanations, this does not appear to be a MailHost Configuration issue. Moving this to the Reporting Help Forum section .... awaiting a reply from some of those folks using the same e-mail client and/for e-mail submittals ....

As a result of a fairly lengthy and intense investigation of Outlook 2003 and 2007: Outlook does *not* include full and accurate headers when you forward spams as attachments. It reorders the Received headers, which makes them untrustworthy, as well as deleting/not forwarding other headers including X-headers, which is of less importance but which may loose some valuable information needed by ISPs/hosting companies.

The result of the 'scrambled" or reordered Received headers means that SpamCop does not reliably know where the injection point of the spam is.

Outlook is reordering the headers, not SpamCop.

Thusly, if you are running Outlook you *may not* forward your spams as an attachment for processing. You can copy/paste or look into running mailwasher or some other 3rd party add-in/add-on but you must stop forwarding as an attachment.

I want to thank the SC users who cheerfully gave of their time ito help in tracking this down.

Ellen

SpamCop

wazoo/mods -- if you would propagate this info to the wiki or other areas as necessary it would be appreciated.

[Spamnophobic]

Do the unfortunate senders of such attachments get a clear explanation back of the reason for rejection?

[Wazoo]

Do the unfortunate senders of such attachments get a clear explanation back of the reason for rejection?

What is it that might not yet be 'clear' based on the flow of the Discussion referenced in Farelf's post if that post doesn't go far enough? What else needs further explanation beyond all the references offered up in the Pinned Announcement at http://forum.spamcop.net/forums/index.php?showtopic=10247 ????

[Lking]

Do the unfortunate senders of such attachments get a clear explanation back of the reason for rejection?

IIUC a new front end for the parser is being developed to identify the senders app, reject the attachments and return an explanation for the rejection.

[Spamnophobic]

If I submitted a test forward, would that be seen as an honest test or as a "falsified report"? As I have explained before, I don't allow any spam etc. on to my computer so I can't forward a real spam from Outlook.

[Wazoo]

If I submitted a test forward, would that be seen as an honest test or as a "falsified report"? As I have explained before, I don't allow any spam etc. on to my computer so I can't forward a real spam from Outlook.

If you don't have spam, can't use Outlook, what exactly is your question? Or of course, an answer to my last question might work also .... what's still missing from all the referenced data?

[dbiel]

If I submitted a test forward, would that be seen as an honest test or as a "falsified report"? As I have explained before, I don't allow any spam etc. on to my computer so I can't forward a real spam from Outlook.

Unless you are talking about quick reporting, forwarding the message is only the first step. There are no penalities or problems if you cancel a parse. The problems start went you click on "Send Reports"

[StevenUnderwood]

If I submitted a test forward, would that be seen as an honest test or as a "falsified report"? As I have explained before, I don't allow any spam etc. on to my computer so I can't forward a real spam from Outlook.

We are going to have to wait for the new interface to be developed before we can know how it is going to work. It may reject just when forwarded which means even test messages will fail when you forward them. It may accept the forward then complain when you try to send the reports which means you could send test messages, though I don't know why you would. You will (as far as we know at this point) be able to copy/paste spam at any time. That is how I am currently reporting.

[Miss Betsy]

At this point, until the new stuff comes in, IIUC, you get a message that the parser can't parse the header (I forget at this moment what the exact message says). You won't know that it is due to the Outlook problem from the rejection message, ISTM. If you come to the forum, and if you know how to search, you will find out that that rejection message is tied to Outlook. If you look around further in the forum, you will find the information that you can no longer forward messages using Outlook. As Steven Underwood says, you can still cut and paste, possibly (again, I forget all the discussion, but ISTM that it is the forwarding that mangles the headers).

What will happen in the future, I don't know. Obviously, the first change that should happen is to edit the 'official' spamcop pages on how to obtain 'full headers' and send 'as attachment' Possibly, the only reference will be in the forum (which is no longer easily accessed from the spamcop help pages) or the ngs (again, not easily accessed from the official pages). Your guess is as good as mine since neither one of us is an 'official' of spamcop.

Also, I, too, don't see the point of sending test messages if you don't receive spam you can report.

Miss Betsy

[Wazoo]

Obviously, the first change that should happen is to edit the 'official' spamcop pages on how to obtain 'full headers' and send 'as attachment'

That was already done, link is peovided in the Announcement I posted in the Announcement secton ... one of those items I've been asking about in the "what's missing" phrases. See Outlook 'Foward as Attachment' no longer authorized !!

Possibly, the only reference will be in the forum (which is no longer easily accessed from the spamcop help pages) or the ngs (again, not easily accessed from the official pages). Your guess is as good as mine since neither one of us is an 'official' of spamcop.

Agree with all that ... a very strange situation, to put it nicely.

[Miss Betsy]

Since I don't use Outlook, I didn't really follow the discussion that carefully. Re-reading the discussion it seems that you don't get a rejection message all the time. Often what happens is that it points to your host. If you have mailhosts, it may not point to your host all the time thus alerting you to the fact that something is wrong with the parse. If it stops at another incorrect point, you may not notice that the parse is incorrect and that if you send the report, you will be reporting the wrong IP address.

Assumption is that deputies pay attention to the test message asking to be allowed to do quick reporting and deny quick reporting because you are using Outlook. I don't remember how VER works - whether you have to be approved or not. The parser, of course, cannot identify an Outlook spam submission since it works on the /attachment/. However, the headers of the email to which it is attached, possibly could be identified as coming from Outlook by X headers though no one found an example.

The official spamcop page is a little confusing about whether the add-on programs can still be used.

Outlook does not properly forward mail with the headers and message body intact. It is not possible to use SpamCop's email submission system with Outlook unless you use one of the below add-on programs or similar macro.

Five paragraphs away

Email submission system

It is not possible to use the email submission system with Outlook. The add-on programs mentioned above will not work with Outlook. You must use the web form to submit spam if you use Outlook as your email client.

I don't understand why they did not delete 'unless you use one of the below add-on....' if they will not work. The only reason I can see is that they do still work on earlier versions, but not Outlook 2007. The alternative is, as Farelf suggested, that the add-ons no longer work with the new version of the parser. In which case, why did they retain mention of the add-ons at all? Why didn't they say, "Add-on programs were developed by spamcop users for previous versions of Outlook, but no longer work properly."

However, in Ellen's announcement, she also mentions that mailwasher or other add-on programs may allow you to forward spam. I don't know how mailwasher works, but apparently it captures the spam email before Outlook so actually you are forwarding from mailwasher (which doesn't mangle headers) rather than Outlook. Perhaps, other add-ons could work the same way, but not the ones developed earlier (which were revised in 2004 to work with a new spamcop parser version).

Is that 'what's missing'? If so, that answers both the question of 'Do Outlook users know that they may be submitting incorrect reports?' answer - no and the reason for the test message - to see if spamcop can identify an email from Outlook - the answer seems to be that, at this time, it will not since it only examines the attachment (which doesn't come from Outlook and Outlook doesn't add anything, just scrambles the header lines).

The person who edited the official spamcop page has 'loosed' a lot of questions even though a clear understanding may be lost. Hopefully, some of that 'loosed' valuable information that ISPs need will find its way home by a new add-on.

Miss Betsy

[David40]

The notice says "all versions of Outlook". Would Outlook Express be considered an included version of Outlook?

[Wazoo]

The notice says "all versions of Outlook". Would Outlook Express be considered an included version of Outlook?

No. Not even sure why this question would be asked after all these years, not to mention the numerous existing FAQ and How-To entries that show many differences between the two applications.. There is no connection between Outlook and Outlook Express much beyond both being a Microsoft product.

[MarkB]

There may be a way to get this fixed with a "grass roots" effort. Ask the Spamcop users to request a Microsoft fix with a unified voice. Post the following request/instructions along with the outlook forwarding announcement and see what happens.

1) Follow this link:

http://www.microsoft.com/office/community/...64-91038e3eb1e9

You will be greeted with this message:

Frustrated? Got a great idea? Want to see a new feature?

The Microsoft Community is a public forum where you can submit suggestions for Microsoft products and see suggestions that others have made. Community participants vote for suggestions, and Microsoft uses the votes to help prioritize features in upcoming versions. Each month, Microsoft will respond to the suggestions with the most votes.

Start by entering your suggestion below. We'll search for it in the community and if your suggestion already exists, you can add your vote to it instead of posting it again.

2) Enter this suggestion:

Outlook should not delete or reorder headers.

3) When asked for additional details enter something like this:

Outlook 2003 and 2007 make it very difficult to trace spam messages because it reorders the received headers, which makes them untrustworthy, as well as deleting other headers including X-headers, which may lose some valuable information needed by ISPs/hosting companies.

4) And finally... ask everyone you know to post to this topic as Microsoft is promising to respond to the suggestions with the most votes.

[rconner]

And finally... ask everyone you know to post to this topic as Microsoft is promising to respond to the suggestions with the most votes.

While you are at it, you can also suggest that Outlook stop (1) relocating line breaks and (2) clobbering the MIME structure of messages. These problems are what requires us to use a separate interface for reporting mail received by Outlook/Exchange.

-- rick

[mrmaxx]

Question -- I've got SpamSource 4.0.10.98 installed on my Outlook 2007. Does that fix the problem or is it still forbidden to submit via email from Outlook?

Since I receive my work email on another computer that doesn't use Outlook as well as my main machine which *does* use OL2007, I can submit from that machine, if necessary, but it would be nice to be able to submit from my desk at work. :-)

Note: I have compared the results of two spam messages I received at work and initially used SpamSource to forward to SpamCop from my Outlook 2007. At a glance, the results are identical. I *did* cancel the ones I forwarded using SpamSource to be sure I did not violate any rules, however I did end up reporting them using "forward as an attachment" from my linux box at home.

Further, reports can be reviewed as follows -- Outlook version (NOT reported): http://www.spamcop.net/sc?id=z3315769724z3...f7f64272f63645z

Non-Outlook version: http://www.spamcop.net/sc?id=z3315804596z3...ca12e2a134942ez

I'd give you the other spam, but I cancelled it before I parsed it so SpamCop does not have any traces of it from the Outlook version.

Edited September 14, 2009 by mrmaxx

[Farelf]

Question -- I've got SpamSource 4.0.10.98 installed on my Outlook 2007. Does that fix the problem or is it still forbidden to submit via email from Outlook?

Since I receive my work email on another computer that doesn't use Outlook as well as my main machine which *does* use OL2007, I can submit from that machine, if necessary, but it would be nice to be able to submit from my desk at work. :-) ...

Hi mrmaxx. I suspect anything, touched by Outlook and submitted by mail, will be er ... suspect but I think you will have to ask a deputy. The door was certainly 'left open' for add-ons/add-ins but AFAIK there has been no confirmation of specifics. Unless anyone comes up with a better idea, I suggest that you ask.

I can certainly confirm that the headers - as seen in the two reports you gave links for - are, character-by-character from the text of the "View entire message" pages pasted into Excel, precisely the same except the Outlook case has two extra (non-critical) lines:

• X-Vipre-Scanned: 0007FA8A00150D0007FBD7-TDI
  
• Bcc:
  

Deputies address is deputies[at]admin.spamcop.net If SpamSource 4 is confirmed as an 'authorised' add-on to Outlook 2007 for mailed submissions, it would be appreciated if you could pass the word on with a further post 'here' . Or if it's not .

[mrmaxx]

Hi mrmaxx. I suspect anything, touched by Outlook and submitted by mail, will be er ... suspect but I think you will have to ask a deputy. The door was certainly 'left open' for add-ons/add-ins but AFAIK there has been no confirmation of specifics. Unless anyone comes up with a better idea, I suggest that you ask.

I can certainly confirm that the headers - as seen in the two reports you gave links for - are, character-by-character from the text of the "View entire message" pages pasted into Excel, precisely the same except the Outlook case has two extra (non-critical) lines:

• X-Vipre-Scanned: 0007FA8A00150D0007FBD7-TDI
  
• Bcc:
  

Deputies address is deputies[at]admin.spamcop.net If SpamSource 4 is confirmed as an 'authorised' add-on to Outlook 2007 for mailed submissions, it would be appreciated if you could pass the word on with a further post 'here' . Or if it's not .

Yeah... I've got a new antivirus I'm testing. It's Vipre from SunbeltSoftware. I'm not sure where that BCC line came from, but... anyway, I'll copy my post and email it to the deputies for review. I suspect they will want more than one sample before giving SpamSource their blessing.

[mrmaxx]

FWIW, Don D'Minion has given his blessing to SpamSource. Of course, it's still a good idea to double-check everything before hitting "send reports." Still, I thought I'd pass it along.

Moderator edit 12-3-09 - Actually Don has NOT given his blessing to any third party tool that would attempt to fix the problem with Outlook's failure to maintain proper headers, but he did provide a list of third party tools that try to address the problem. see Here is the boilerplate on Outlook...

Note: discussion that follows has not been edited, but this initial post was edited to avoid "a misrepresentation of what was actually said" - dbiel -

Edited December 3, 2009 by dbiel

[turetzsr]

FWIW, Don D'Minion has given his blessing to SpamSource.

<snip>

...Thanks, mrmaxx! Edited December 14, 2012 by turetzsr

[Farelf]

FWIW, Don D'Minion has given his blessing to SpamSource. Of course, it's still a good idea to double-check everything before hitting "send reports." ...

As later noted in Don's Outlook Boilerplate.

[mrmaxx]

Not to be flogging a deceased equine, but when I went into my SpamSource configs, I noticed there was a button marked "repair" and upon clicking on that, it gives some options for repairing mangled messages, such as reformatting headers, etc. Should any of those options be enabled?

[Farelf]

Not to be flogging a deceased equine,...

No, no, such clarification is relevant.

...but when I went into my SpamSource configs, I noticed there was a button marked "repair" and upon clicking on that, it gives some options for repairing mangled messages, such as reformatting headers, etc. Should any of those options be enabled?

I shouldn't think so, no. Your received headers (those in the spam in your junk/inbox) are your received headers and the problem with Outlook is not there - it is when it is used to forward those headers and then, under some circumstances, it re-orders those received headers. The 'test', when people were researching the Outlook problem, was to see if there was a difference between the received headers forwarded and those, erm..., received.

I am not sure what function the SpamSource "Repair" configurations might serve or how they might work or where the source of the damage they anticipate might be deemed to have occurred - but as long as the headers in your SC submission precisely match those that are revealed in the source of the original in your junk/inbox (which are deemed the "authentic" headers) then you don't have to use them, maybe you shouldn't use them even, in the event they might add some unexpected and bogus alteration.

Of couse we would all feel rather more confident about this if we could hear from an experienced SpamSource user on the subject. Theoretical pontification is one thing but the experience of a knowledgeable user is more to the point.

[CGretski]

Does Exchange 2007/2010's Quarantine suffer from the same issue, spam is presented to Outlook as an NDR with send-again to access the original message. (It's possible to save the original email to disk as an attachment via VBA and then add that attachment to a new message - is there a definitive test case to see if that messes up the headers?)

Also, do any of the suggested programs have the ability to process said NDR to get the original spam message (spamsource silently failed).

Thanks

[Farelf]

Does Exchange 2007/2010's Quarantine suffer from the same issue, spam is presented to Outlook as an NDR with send-again to access the original message. (It's possible to save the original email to disk as an attachment via VBA and then add that attachment to a new message - is there a definitive test case to see if that messes up the headers?)

Also, do any of the suggested programs have the ability to process said NDR to get the original spam message (spamsource silently failed).

Hi CGretski,

I think you will need to liaise with Don (SC Admin) on this - service[at]admin.spamcop.net

As you will have noted, Outlook sometimes shuffles the order of the "Received:" lines when there are multiple occurrences. Definitive tests have shown this is an unacceptable risk for the reliable identification of the source. It *sounds* like the handling you talk of might be safe but I just don't know. Other members of the forum will be better able to comment but if your reports ride in on Outlook, Don will need to be happy with the process. We have expert Exchange users and hopefully one of them can contribute something a little more meaningful to assist you.