Jump to content

[Resolved] Attached sent spam gets truncated on occasion


Recommended Posts

Lets not get mired in linguistics and tone of voice. We must assume that when deviantchild reports that he used the <write> function to start a new email that is what he did. When you drag and drop with TB the only result is a attachment (I tried you can't get it inline as in <forward>).

OK, then can you explain why the example presented here from inside of spamcops reporting web page is showing the inline tag AND headers showing the message being sent from the ISP to the reporting address?

I give up because I am not getting the answers needed to answer the original query.

Link to comment
Share on other sites

The fact you are NOT following the defined steps in the FAQ only leads to more questions to be answered.

The FAQ is out of date:-

Edit > Preferences

Composition Tab 'Forward messages: as attachment'

Is now:-

Tools > Options

Composition page > General tab 'Forward Messages: As Attachment' (as opposed to 'Inline')

Advanced Tab, Privacy 'Block loading of remote images in mail messages'

Is now:-

[from where we left off]

Advanced page > General tab > Config Editor button (which brings up About:Config)

locate the entry 'mailnews.message_display.disable_remote_image', where default settings are:- Status: Default; Type: Boolean; Value: True

Please stop presuming. Especially if you have never used Thunderbird. It was the excellent Bayesian filtering and security which has maintained my preference. FYI, the default setup in Mozilla products generally adhere very well to standards. Preferences in these products are more there to allow user control and manipulation, to encompass unusual requirements. You generally shouldn't need to alter things unless you require something a little more tailored and specific.

And yes, upon getting past that first part and reading the rest of the FAQ again, it seems I am doing as it suggests. Although they do list the three methods of achieving attachments:-

1.1) Edit > Select All

1.2) Ctr +A

1.3) Select the first, scroll to the last, holding the 'Shift' key down click on the last to select all from first to last.

...of which, I prefer the third.

I still maintain that all should perform the same function when standards are followed, as I believe is the case here.

You'll be pleased to hear however, that a discrepancy arises with the following:-

2) Right-click on the selection and choose 'Forward as attachments'.

This will open a new email with all selected spam emails included as attachments.

Doing it this way lists all the attachments by name, followed by the e-mail extension .eml

Whereas...

Pre-composing a blank e-mail, then dragging&dropping still lists them individually, yet each one is referred to as an 'attached message' rather than how stated above!

I don't quite know how to interpret that one, but it still doesn't explain how, only when the mail has been truncated* it has spewed out in one long, seemingly inline message with me fingered as the spam culprit. Although, in other data, when a CRC check fails it never stopped a person from viewing the raw data that remains. In the case of ASCII (or similar) it's still going to look how it was intended.

Anyway, I feel I'm clutching at straws there and that is why I came to the forum instead - because you guys know more about mail systems and protocols than I. Ignore my wild assumptions in this respect. They're just me thinking aloud, and not to be seriously considered.

*Also, don't get me wrong. This hasn't happened often, but it has been annoying when it has. I do a few submissions per day, and all are usually fine.

Link to comment
Share on other sites

Yes, it is tiresome, but in order to find the problem, details are important. What may seem to you as something that 'should' happen and not worthy of comment may the one place where it makes a difference what has been done.

Crossed wires again, I'm afraid.

I [like to think I ;) ] understand the troubleshooting process somewhat as I used to work and frequent several graphic card forums.

Much of what you go on to say is indeed valid, and worth general consideration.

Yet, I was [somewhat sarcastically] referring to getting all tied up with explaining myself, because I perhaps hadn't covered all bases with my choice of words earlier, rather than getting on with the job in hand.

Bottom line: if you want to know what is wrong, then tiresomely picking at the details is the way to find out.

Miss Betsy

Correct. Yet I do feel I have been scrutinised here as a newbie computer user, which I am not. As opposed to a newbie spam reporter, which I freely admit I am; and thus reliant entirely upon you guys for understanding of the technical side of that.

Thanks anyway, and no hard feelings.

Link to comment
Share on other sites

OK, then can you explain why the example

No, but after using the same tools he uses, trying the same procedure he described, and verifying the the results he reported (except the email to SP), I did suggest some other possible sources of the error (and I think) without insulting his intelligence.

I give up because I am not getting the answers needed to answer the original query.

Sorry to hear that Steven. I try to speak only about what I know. I do not know much about ISP operation (except as a whiplashed user) or using Hotmail (which he reported). It would be nice if others with knowledge/experience in these area could ...

Link to comment
Share on other sites

Well, I hope that s/he tries to submit that peculiar spam singly or via the web form and see if the same thing results. Or tries forwarding it to another account and see what happens to it.

Was I wrong that there was no tracking url? "..why the example presented here from inside of spamcops reporting web page is showing the inline tag AND headers showing the message being sent from the ISP to the reporting address..." I thought that example was from the sent mail folder.

If Lking thinks that the submittal method is correct, then it has to be something that someone does to it on the way because of some characteristic that distinguishes it from other email that has no submittal problems.

Or possibly Tbird handles it differently because of something in the email.

Or it is one of those emails that spamcop chokes on unless you have configured mailhosts (the fact that it pointed to hir as spammer).

Miss Betsy

Link to comment
Share on other sites

Was I wrong that there was no tracking url? "..why the example presented here from inside of spamcops reporting web page is showing the inline tag AND headers showing the message being sent from the ISP to the reporting address..." I thought that example was from the sent mail folder.

The example was copy & pasted here from inside SpamCop's reporting page.

Link to comment
Share on other sites

...Was I wrong that there was no tracking url? "..why the example presented here from inside of spamcops reporting web page is showing the inline tag AND headers showing the message being sent from the ISP to the reporting address..." I thought that example was from the sent mail folder.
The example appeared to have been a copy of the "full message" page from a parse, the text of the example began "CLICK 'BACK' BUTTON TO RETURN TO SPAMCOP". It appeared to show a succession of spam, each within its own mime boundaries and thus appearing to the parser as a single spam (and therefore subject to arbitrary truncation at 50k, not to mention that no attempt would be made to track the several sending sources since this would be all body as far as the parser was concerned). If this example was sent with the spam items as attachments, the result is indistinguishable from "inline" content when it is received by SpamCop. The actual tracking URL was not presented.

Whatever the problem is, it must be "solvable". It seems exactly like a common problem but has thus far eluded any "Eureka!" moment because the same submission method is said to produce variable results. [Added] - well of course the results *would* be variable when truncation occurs because the ending mime boundary is then missing so the parser would always show an error. When truncation does not occur the parse could not possibly handle the inline headers. No evidence has yet been given of successful submissions. Submissions below 50k with inline spam *could* appear successful because they don't bomb out. But of course they're not successful.

Link to comment
Share on other sites

Had one go awry this morning.

Returned this error: error: unexpected end of header error: couldn't parse head

Which I briefly read, but got a bit fed up with being told that it was me messing around that probably caused it:-

...It is an error introduced by the recipient (you) when copying or submitting email to spamcop. If you encounter this problem, please review how you submit spam to SpamCop and take corrective action.
My fault again! *sigh* ;)

Some details on the sent mail:-

1) It was created by using Shift+Cursor selection method (from within my 'Junk' folder) and then right-click context option to "send as attachments"

2) Although the Deviantchild e-mail address was used in the address field, my Blueyonder SMTP was actually used to send it.

3) The entire mail (as reported in my 'Sent' folder) turns out to be 119kb - which was a slight accident, yet others far larger have been accepted at the other end just fine. I am trying to keep them below 100kb, but I'm glancing over the sizes and not sitting there with pen and paper tallying them up! ;)

I have supplied the two files: the original source and the SpamCop result for comparison HERE (you will need to re-add the .rar extension before you can extract them - small bit of privacy for me)

I used ExamDiff to view them. It's clean and simple, yet performs text comparison nicely.

Thanks.

Edit: Sorry. Here's the Tracking URL

[Moderator edit - pending report cancelled. Presumably you don't want some funster reporting blueyonder for you.]

Link to comment
Share on other sites

Unfortunately, the error that is returned has, in the past, always been the result of the way that the reporter submits the spam. It is not directed at you personally. For some reason, the spam is causing this error in spite of the fact that it is being submitted the same way that successful parses are.

There are no headers that show where the spam originated from. The headers should have where the spam originally came from, then that it was forwarded to the blueyonder account from another account where it was received (if it was - I am assuming that) and that's all except for spammer forgeries and X-lines added along the way, IIUC. That's the point of forwarding as attachment so that the original headers remain intact. The only time that blueyonder would be in the picture as the source IP is if the spam came from within the blueyonder system to you . which could be the problem since the blocklist that that IP address is in is one for zombies, I think.

It seems as though something is stripping the headers the way that Outlook does. Are the headers of the original spam just like the tracking url? (Sorry, I don't do fancy extensions, etc. - told you I am technically non-fluent).

Have you tried submitting the raw message source of the spam itself via the web form? Have you tried submitting just that spam? And what are the results? tracking urls (copying to another place always has the potential of creating lines, spaces that the parser didn't see).

Miss Betsy

Link to comment
Share on other sites

Had one go awry this morning.

Returned this error: error: unexpected end of header error: couldn't parse head

Further to Miss Betsy's sage (as always) commentary I would add that I have little doubt we are looking at your submissions *in-line* rather than attached - and as I commented on the basis of your initial example (which I removed at your request) when that happens the parser treats the whole submission as a single email, doesn't look in the "body" for the actal spamsources but just truncates at 50k. Therefore the closure of the MIME boundary doesn't happen and you get that error message.

Now I know you are submitting correctly. Not that I use Thunderbird but I use Mozilla and as you will appreciate there is a lot of common development and the controls, menus etc. are similar. Somehow T'bird is letting you down. There was some discussion on the vagaries of the beast back in Thunderbird forward as attachment works [at]50% and there may be other "history" to be found in these pages. Different symptoms but casting doubt on the ability of T'bird to do as it is told. Having said that, there are other ways you can forward as an attachment (via the menu, rather than right-click). You should also try that.

But I have no (well, little) doubt your truncated cases are due to inline attachments. The danger is, some collations of smaller spam may be scaping through in this form as well, without generating an error.

Now if you are sending submissions through Blueyonder you must nominate it as a mailhost. That way you can't inadvertently report them for forwarding your submissions. The worst that will happen when mailhosts are properly configured is you would get a "Nothing to do" message if the Blueyonder is the only IP address in your headers (that is when the attachments are in-line).

HTH

Link to comment
Share on other sites

[sorry. Slightly OT]

I've just added some mailhosts:-

The Zoom one only got sent via one server

The Blueyonder got sent via three

The Hotmail got sent via four

The Zoom one arrived and got forwarded to the addy provided

The three Blueyonder ones arrived and got forwarded back to the addy provided

The Hotmail ones haven't turned up yet and, upon checking online, are not in Hotmail's webmail Junk or Trash folders ???

The Zoom one returned an error, due to it having no headers, shortly followed by a success!?

The three Blueyonder ones also return both error (for the same reason) and a success for each one!?

I also tried forwarding the troublesome submit from earlier to myself, from Blueyonder to Zoom.

This too has disappeared and is also not to be found online, in either their webmail SpamFilter folder or Trash folder?!

Could this be because I grassed myself up to Blueyonder as a spammer previously?

LOL

If you want something doing, do it yourself! ;)

http://kb.mozillazine.org/Send_attachments...eal_attachments

One for the FAQ, I feel.

Strange though, how there is the previously listed visible option to choose between Attachment or Inline, yet the switch listed in that piece, and found under About:Config, was set to '0', and therefore mustn't be related somehow. Either that or TBird V2beta2 is slightly broken somewhere.

Link to comment
Share on other sites

LOL

If you want something doing, do it yourself! ;)

http://kb.mozillazine.org/Send_attachments...eal_attachments

One for the FAQ, I feel.

Strange though, how there is the previously listed visible option to choose between Attachment or Inline, yet the switch listed in that piece, and found under About:Config, was set to '0', and therefore mustn't be related somehow. Either that or TBird V2beta2 is slightly broken somewhere.

So much for configuration control!

I just checked my version of FB, 1.5.0.10, and mail.content_disposition_type is "0" unlike the suggestion by MozillaZine which suggest "1".

This am I ran the following test:

forwarded 2 spam using the right click, "Forward as Attachment" works fine Submit URL 1 Submit URL 2

Forwarded 3 spam using the drag and drop approach and quick reporting, also works fine Quick URL 1 Quick URL 2 Quick URL 3

And Forwarded 1 spam using the right click, "Forward" (their words, which also forwards an attachment) Quick URL A

I think the clue is, from the MozillaZine link, "Some recipients will see some of your attachments as part of the message body". And the fact that deviantchild uses several mail services. Each mail service may "see" the attachments differently, process it differently, resulting in different email being sent to SC.

Hope this adds so someone can teas out an answer.

Link to comment
Share on other sites

Lking: appreciate the effort, but ... all of your samples were a single spam, whereas the 'complaint' is about "multiple spams in a single e-mail" ...

Deviantchild: .. so much ground to cover, and based on past experience, you'll get ticked if I try to address even a few of them ....

Your last inclusion of your use of a "Beta" ... ahem ... that's part of what "Beta" means ... you have allegedly volunteered to be a troubleshooter, offering to find and report things found broken, in hopes that they can be fixed prior to releasing the 'final' version.

As seen in the general discourse here, the focus "is on you" as there are so many other Thunderbird users floating around without the problems you are having ... though of course, wondering why these same folks aren't kind enough to jump in here with their additional data, be it configuration settings or procedural advice (though again, it is possible that the other discussions, FAQ entries, etc. worked fine for them???)

What I'm seeing at this point is that you need to do some troubleshooting on your own. As noted and exampled, your samples (though noting a lack of Tracking URLs for either successful or failed parses can't be overlooked) are all showing the "in-line" condition .... even Miss Betsy has noted issues with the headers involved in your last sample (that Farelf provided a Tracking URL for) had serious issues as for as being a "technically correct" submittal.

Now that you have found a FAQ entry on the tool in use, have you done anything with that data?

Suggestion would be to send yourself some e-mail with several attachments, from one hosted account to a different hosted account, view the source of the received e-mail, looking for the condition of how those attachments actually arrive.

In deference to Lking's suggested possibility, an e-mail server should not get its hands dirty by playing with the headers of a traversing e-mail, changing the RFC-attachments to 'inline' data .... Lord knows what else would be 'tampered' with on an e-mail server that took these liberties.

Bottom line, the "problem" has been identified. How to fix it appears to be on you at this point.

As far as "the FAQ is out of date" ... well that goes back to how it is generated. The item referenced that you remarked on was a post provided by yet another SpamCop.net user .... copied by a Moderator and placed into the How to use .... Forum section such that it could be found by someone actually trying to find that particular data. That you 'complaints' aren't in that Topic doesn't help bring that entry up-to-date. Worst case, an additional post wit new data based on the different version of the tool you are now using would be appropriate (of course, after you sort out what the actual problem is and how to get it to work 'correctly')

Link to comment
Share on other sites

Lking: appreciate the effort, but ... all of your samples were a single spam, whereas the 'complaint' is about "multiple spams in a single e-mail" ...

Deviantchild: .. so much ground to cover, and based on past experience, you'll get ticked if I try to address even a few of them ....

Your last inclusion of your use of a "Beta" ... ahem ... that's part of what "Beta" means ... you have allegedly volunteered to be a troubleshooter, offering to find and report things found broken, in hopes that they can be fixed prior to releasing the 'final' version.

Yes, if it was a MS product I'd be severely concerned, as they tend to rewrite a lot of stuff and cause more issues than they resolve, due to conflicts with other dependencies. However, TB betas, in my years of experience using Mozilla stuff, are more like subtle point releases. They are usually referred to as 'daily builds' and not much is altered, except for minor submitted bugs being ironed out. More a kind of slow evolutionary process, rooted in common sense and general preference, which I like a lot.

As seen in the general discourse here, the focus "is on you" as there are so many other Thunderbird users floating around without the problems you are having ... though of course, wondering why these same folks aren't kind enough to jump in here with their additional data, be it configuration settings or procedural advice (though again, it is possible that the other discussions, FAQ entries, etc. worked fine for them???)

Why keep harping on about the FAQ? Once I'd gotten over the 'out of date' discrepancy at the start, it turns out standard procedure is, and always has been, the same. FYI, I don't screw about with preferences if a) I don't understand what they relate to and B) thus, would probably deviate from the norm and break something. My setup is default.

What I'm seeing at this point is that you need to do some troubleshooting on your own. As noted and exampled, your samples (though noting a lack of Tracking URLs for either successful or failed parses can't be overlooked) are all showing the "in-line" condition .... even Miss Betsy has noted issues with the headers involved in your last sample (that Farelf provided a Tracking URL for) had serious issues as for as being a "technically correct" submittal.

<sarcasm>Tell me what's not "technically correct" then? is it the 'attachment' function or the 'send' function? I can remove one or the other, but if that's the case I think I'll struggle to complete any more submittals without either! </sarcasm>

STOP ASSUMING I'VE MESSED WITH MY SETUP

Now that you have found a FAQ entry on the tool in use, have you done anything with that data?

What "tool"? Thunderbird? Of course I did. What do you expect? "No, I found a potential solution so decided to do nothing"... WHAT?!

Suggestion would be to send yourself some e-mail with several attachments, from one hosted account to a different hosted account, view the source of the received e-mail, looking for the condition of how those attachments actually arrive.

Read my previous post. I did!... and one of those test mails completely disappeared off the radar!???

In deference to Lking's suggested possibility, an e-mail server should not get its hands dirty by playing with the headers of a traversing e-mail, changing the RFC-attachments to 'inline' data .... Lord knows what else would be 'tampered' with on an e-mail server that took these liberties.

So, if the server didn't do it, and I didn't do it (unless I don't realise I did, because I am a schizophrenic pathological liar) then what did?

Bottom line, the "problem" has been identified. How to fix it appears to be on you at this point.

If you'd have read my previous posts (and also now here) you'd have realised I'd started taking steps - one at a time. You cannot change two factors in an equation at once else you'll potentially never get an answer - my own golden rule of troubleshooting, from having been a sound and lighting technician.

As far as "the FAQ is out of date" ... well that goes back to how it is generated. The item referenced that you remarked on was a post provided by yet another SpamCop.net user .... copied by a Moderator and placed into the How to use .... Forum section such that it could be found by someone actually trying to find that particular data. That you 'complaints' aren't in that Topic doesn't help bring that entry up-to-date. Worst case, an additional post wit new data based on the different version of the tool you are now using would be appropriate (of course, after you sort out what the actual problem is and how to get it to work 'correctly')

That was a response to quick and dirty finger pointing methods that I encountered upon arrival. "Look, you're not following the FAQ" To which I retort "It's out of date" and "there are bleeding design standards and norms that evolve, and are retained purely because people expect and know them" I really don't expect Mozilla to start breaking stuff in order to create and monopolise said standards in the way that I expect corporations vying for patent controls and subsequent paying licenses do.

Yes, you were right. I do get "ticked". But, get this: I don't hate you all - until, that is, we have gone so far off track and I tire of saying "Look, I am not altering or messing with anything here. Give me a break!"

ps. If many of you aren't using Thunderbird, and I presume wouldn't be using Outlook(/Express) with this being an anti-spam/security based site. Then, what non-bloatware, non-loophole riddled software is alternatively available, with good Bayesian (or better) filtering? I've heard of Eudora, but never tried it as I've been happy with everything Mozilla have ever achieved.

Link to comment
Share on other sites

Lking: appreciate the effort, but ... all of your samples were a single spam, whereas the 'complaint' is about "multiple spams in a single e-mail" ...

Wazoo: what I must have failed to make clear was that using one email I sent 2 spam, using one email sent 3 spam and using one email I sent 1 spam. Of course when all is working correctly that results in 2, 3, and 1 tracking URL. Deviantchild is doing the same BUT getting a single tracking URL because ... as you documented before. My point was that with my version/config of FB the process described works.

Your last inclusion of your use of a "Beta"

That "little" point I missed. You are correct, all bets are off. Without that info I jump to the service (which has seemed to be a big variable when dealing with spam). Your prospective is correct, why would they bother to mess with the header? I guess if you are really paranoid...

Updating to the current release of FB is an answer so we are all working with the same tools, and you can't beat the price.

Link to comment
Share on other sites

No one is saying that you have deliberately configured your computer so that it does this, but it almost has to be something in the way it is configured to make it behave this way.

Since it is multiple spams that cause this problem, then each one should be submitted and the report cancelled to see which one is causing the problem. Since Lking has pointed out that some email services 'see inline' could it be that BlueYonder is 'seeing' a forwarded spam as 'inline forward' thus messing up the whole submission? Or possibly that wherever it is forwarded from uses inline forwarding by default?

The only way I can see to pinpoint where the problem is, is to submit each spam separately and see what the results are. If any one causes a problem, then submit the original spam from wherever it was received and see what happens then.

Or maybe you are just going to give up on Thunderbird and try another.

Miss Betsy

Link to comment
Share on other sites

Just sounded like people figured I had configured something out of the norm, despite me consistently telling them not. I don't do that, and I don't lie. I am the guy round here who gets to fix everyone's computers. I generally know how to get the best out of systems. And that means not screwing with stuff on the software side that other software may generally assume (unless specified) is set up in a default manner.

No, I won't give up on Thunderbird. I don't think I want to go changing to Eudora as, until this little episode here, TBird (and other Mozilla applications) have performed fantastically over the years.

If you read back, you'll find there is a 'blip' which means Thunderbird is sending inline. Which is normally great for most purposes. But, now reading up on it, it seems that it is acknowledged that there are small, differing issues with sending one way, or the other as true 'attachments'. As is apparent with SpamCop's requirement.

If you also read back you will find that by using about:config (a kind of configurator console, for those not versed) I seem to have suppressed the issue by using a switch. I hasten to add that, the unwitting 'inline' sending method only threw up an issue in possibly 1/10 / 1/20 submits to SpamCop.

Truth is, this anomalie threw us.

The fact that the visible setting "forward messages:" attachment/inline may not 100% refer to the lower level setting which determines how all stuff is treated. I don't know why, and it does seem to need addressing. And, as you know, being inline has caused issue with how my mail has been interpreted along the line from me to SpamCop. As I said, SpamCop hasn't always found issue with the mails, unless they've been truncated. And the possible reasons for that were stated by one of the mods earlier. So, as far as I was concerned, everything was seen to be set to 'attachment', though they weren't. Now they are, and I haven't had a single problem yet.

So again, this time simply put:

An attachment sending method setting (that was visible and set correctly) wasn't being adhered to. The underlying console setting taken from the following link rectified that situation.

http://kb.mozillazine.org/Send_plain_text_...eal_attachments

ps. in the console there also appear to be settings to 'view' messages inline or not, whether or not they were composed that way.

Link to comment
Share on other sites

Now they are, and I haven't had a single problem yet.

Thank you for the complete summary. I hesitate to mark the issue resolved only because of the randomness of the original problem. Another moderator may feel differently and mark it Resolved.

Link to comment
Share on other sites

I am glad that the OP is satisfied that he has finally 'found the problem'

Unfortunately, I don't have a clue what he found except that appartently the default settings (or the sequence) did cause real 'inline' submissions and that whatever it was, the OP has figured out how to avoid it.

But I am still glad that the OP found the solution. Not quite the 'eureka' moment I was waiting for, but close enough.

Miss Betsy

Link to comment
Share on other sites

I am glad that the OP is satisfied that he has finally 'found the problem'

Unfortunately, except that appartently the default settings (or the sequence) did cause real 'inline' submissions and that whatever it was, the OP has figured out how to avoid it.

But I am still glad that the OP found the solution. Not quite the 'eureka' moment I was waiting for, but close enough.

Miss Betsy

I don't quite know what you mean by "sequence" as I have explained that there was a difference in the default setup to what SpamCop requires in order to parse correctly. Unfortunately, with there being a setting showing the program being set to 'send as attachment' rather than 'send inline' I, or anybody for that matter, would have been none the wiser that the program was ignoring it. I only delved deeper because I too had seen the 'inline' tags, even though the program was, for all intents and purposes, seemingly configured not to do so.

I don't understand why there is no eureka. There was a error in the reported state of the application, and it has now been circumvented. That's pretty much spells a eureka moment to me.

Anyway, enough now. It's sorted. The solution has been found. I'll consider informing Mozilla of the discrepancy.

Thank you.

Link to comment
Share on other sites

... There was a error in the reported state of the application, and it has now been circumvented. That's pretty much spells a eureka moment to me. ...
And to me, thanks for your efforts and clarification, hopefully they will benefit other reporters experiencing the same problem. Marking 'resolved'.
Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...