"No body text provided" with Bcc: header

[MIG]

20 minutes ago, euphorique said:

As always, "No body text provided, check format of submission. spam must have body text."

Buggar😔grass🦗hopper!, so the question is, how did RobiBue get it to parse correctly🤔

I know this sounds too simplistic but have you tried the parser using a) different browsers? 

b) Cleared out all cache, cookies, history, maybe even

c) reset broswers you use? [Save all bookmarks, config settings]  before if you do these things.

It doesn't make sense that RobiBue can get it to parse & we can't🤔

[RobiBue]

50 minutes ago, euphorique said:

This one has "validly formed" address in bcc: https://www.spamcop.net/sc?id=z6523551992z5978bb9216921fdbce50eebacde4661az

Received: from mbkd0231.ocn.ad.jp (mbkd0231.ocn.ad.jp. [153.149.233.32])
        by mx.google.com with ESMTP id e96si22718016plb.123.2019.02.21.05.54.06;
        Thu, 25 Feb 2019 05:54:07 -0800 (PST)
From: spammer@spam.com
To: me@mailinator.com
Subject: problem with BCC: header
BCC: spammer@spam.com

Hey! There is a blank line between the headers and the body!

 

while I admit, that the "test" message gets the message body stripped if the Bcc: header line is the last one, I do have a question:

Is there an email software that actually does send/receive the Bcc: header as the last line?

I tried gmail and yahoo. sending from yahoo to gmail, the bcc header was not the last header line and it works.

Sending from yahoo to gmail, the bcc header got removed, so it worked as well.

While I understand that there is a problem with the parser, and believe me, it's by far not the only one, the workaround is to remove the bcc header before submitting.
Since bcc headers are not expected to appear in headers according to rfc2822, it doesn't matter if the spam is reported without it anyway.

 

[MIG]

6 minutes ago, RobiBue said:

while I admit, that the "test" message gets the message body stripped if the Bcc: header line is the last one, I do have a question:

Is there an email software that actually does send/receive the Bcc: header as the last line?

I tried gmail and yahoo. sending from yahoo to gmail, the bcc header was not the last header line and it works.

Sending from yahoo to gmail, the bcc header got removed, so it worked as well.

While I understand that there is a problem with the parser, and believe me, it's by far not the only one, the workaround is to remove the bcc header before submitting.
Since bcc headers are not expected to appear in headers according to rfc2822, it doesn't matter if the spam is reported without it anyway.

 

Thanks RobiBue! Standby please, grass🦗hopper is going to try!

[MIG]

☹️Grrr!, nope!

[MIG]

Hmm, with euphorique's raw data, getting rid of bcc, and other mods, can't get past "Mailhost configuration problem, identified internal IP as source". I think that's the "old" gmail 6 issue that's been fixed in V5, so (for grass🦗hopper) needs to do to mock up my hosts or the data to get past this... 😔☹️

[MIG]

Hey euphorique, 

Based on RobiBue's "there is a problem with the parser", comment, I parser your raw source data with  [ https://testconnectivity.microsoft.com/ ][Message Analyzer]

It's not the same as SC parser but, have a look and see if that helps at all?

Cheers!

[Lking]

4 hours ago, RobiBue said:

Please forgive my ignorance here, but

1. I've never seen receiving headers with bcc in them...
   afaik they get stripped by the sending mail host and never shown to any recepient.

RobiBue makes a good point, that I missed back in May 2018.  There is a difference between submitting case 1,2,3 as text directly to the parser and submitting a received email (as text) to the parser.  As RobiBue noted, when a user sends an email, containing a Bcc address to a properly configured email server, the email system generates two outgoing emails hiding the Bcc: addressee.

In my test just now

1. one email

Quote

From - Thu Feb 21 11:01:11 2019
X-Account-Key: account1
X-UIDL: 1371062169.138342
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
Return-Path: <info@xxx.org>
Received: from DOMAIN.com ([unix socket])
	 by DOMAIN.com (Cyrus 2.5.11) with LMTPA;
	 Thu, 21 Feb 2019 18:00:58 +0000
X-Sieve: CMU Sieve 2.4
Received: from localhost (unknown 
.
.
.
	Thu, 21 Feb 2019 18:00:57 +0000 (UTC)
To: Webmaster@DOMAIN.com
From: info <info@xxx.org>
Subject: Testing
Message-ID: <20f62015-c34b-d1d7-cb5b-80d6d7614873@xxx.org>
Date: Thu, 21 Feb 2019 11:00:52 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:59.0) Gecko/20100101
 Thunderbird/59.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Brightmail-Tracker: AAAAAgr/Be8zuFC8
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrFqMTGxcIABLo1z/NiDGb8UbfYd+A7iwOjx/EX75kCGKM4/PJLFIILEnMTODJ6dl9gK2CoAFENjAyrGOVyS1MNizMSi0sSE4uKk3JKUwuK8lN0y3INwGATI9TsHYzbL8qcYpSUEud9/DQvRkigGGheRmleSWpRfFFpTmrxK0ZxDkYlYd4Fz4CyPJl5JcWZ6TAZCQ4mJRFe43/ZMUK8QIsQUlINjGY7Yx0dNayFXeJNEyS2/XBcxW5oX5+qLn355xmPt1Pe/ZrwL8pD54vGRreP84u77H69jDvstSfai9vxZOD+f3Zq/wtOaCprmnB/7LI68+MzxxkBBbPOGfPSdJ+EHnL9kDPZPJ/50oR9l7o3/tw6zUc3WWf1haeZV89UejjFx179Xvid02xvrBIL0PeGWsxFxYkAZxzFLU0BAAA=

This is a BCC test

was received by the Bcc addressee.  Yes I deleted the routing, Just more IP addresses etc than I want to reveal.

2. the second email

Quote

From - Thu Feb 21 11:01:50 2019
X-Account-Key: account6
X-UIDL: 1368657514.83635
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys: $label4                                                                         
Return-Path: <info@xxx.org>
Received: from DOMAIN.com ([unix socket])
	 by DOMAIN.com (Cyrus 2.5.11) with LMTPA;
	 Thu, 21 Feb 2019 18:00:58 +0000
X-Sieve: CMU Sieve 2.4
Received: from localhost (unknown 
.
.
.
               Thu, 21 Feb 2019 18:00:57 +0000 (UTC)
To: Webmaster@DOMAIN.com
From: info <info@xxx.org>
Subject: Testing
Message-ID: <20f62015-c34b-d1d7-cb5b-80d6d7614873@xxx.org>
Date: Thu, 21 Feb 2019 11:00:52 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:59.0) Gecko/20100101
 Thunderbird/59.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Brightmail-Tracker: AAAAAgr/Be8zuFC8
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrFqMTGxcIABLo1z/NiDK69s7TYd+A7iwOjx/EX75kCGKM4/PJLFIILEnMTODJ6dl9gK2CoAFENjAyrGOVyS1MNizMSi0sSE4uKk3JKUwuK8lN0y3INwGATI9TsHYzbL8qcYpSUEud9/DQvRkigGGheRmleSWpRfFFpTmrxK0ZxDkYlYd4Fz4CyPJl5JcWZ6TAZCQ4mJRFe43/ZMUK8QIsQUlINjPOX1NvF+8RvZFl1NvdhvEwGn4PmFX2+mgyZ5Penguvyzpx04wrf5ye+af6vumx9V2P/ZCkzF9vpD1mimf7F8QfpLmzw4TN0dS7TiVZPkTutmaj7ssW3Oi/4O+Pi9AXT+5tkxC7apB1dLNOZHq7zZP/RP5YcBXfb8oLPHJ/+PXjr4aRFmRFKLEDfG2oxFxUnAgDwz0TQTQEAAA==

This is a BCC test

was received by the TO: addressee.  Note nether email contains a Bcc: line as pointed out by RobiBue, which if included would defeat the purpose of the Bcc:

RobiBue's observation brings us to the point mentioned several time here, 'The parser can not be written to handle every possible improper configuration or error generated by spammers.' This is another example.

Preemptively let me point out that that, yes email applications may be able to handle this improper header. The objective of email applications is to try to deliver to the user any/every email received. If the application is delivered incorrectly, the human reader may be able to use the results.

The SpamCop parser on the other hand, to remain credible, MUST always be correct when sending a spam Report (accusing someone of sending spam). If the parser incorrectly tries to interpret a poorly formatted header and as a result incorrectly accuses someone of spamming credibility is lost.

A trivial example could be: "Let's eat grandma!"   Is that something for the grammar police or the Donner party?

[RobiBue]

As I mentioned in my previous message, I did a test between Yahoo! and GMail with Bcc: header test.

Here the "munged" but true header results:

Outgoing test message with Bcc: from Yahoo!:
-------------------------------------------

Date: Thu, 21 Feb 2019 15:57:40 +0000 (UTC)
From: RobiBue <MyFakeYahooAccount@yahoo.com>
Bcc: "RobiBue" <MyFakeGMailAccount@gmail.com>
Message-ID: <1883948137.3003972.1550764660065@mail.yahoo.com>
Subject: test with Bcc: address
MIME-Version: 1.0
Content-Type: multipart/alternative; 
	boundary="----=_Part_3003971_1789674874.1550764660064"
References: <1883948137.3003972.1550764660065.ref@mail.yahoo.com>
X-Mailer: WebService/1.1.13123 YMailNorrin Mozilla/5.0 (Windows NT 6.1; WOW64; rv:67.0) Gecko/20100101 Firefox/67.0
Content-Length: 505

The above outgoing message (in the sent folder) shows that the Bcc: was there when it left the Yahoo! mailbox.

Next comes the GMail inbox part:

incoming test message in GMail (removed arc seal and other long winding headers in the middle):
-----------------------------------------------------------------------------------------------

Delivered-To: MyFakeGMailAccount@gmail.com
Received: by 2002:a0c:9de7:0:0:0:0:0 with SMTP id p39csp979189qvf;
        Thu, 21 Feb 2019 07:57:44 -0800 (PST)
X-Google-Smtp-Source: AHgI3IZpr/3XS19bX4AwkcuhfNfYJk7Gk3+ZDjldlb8JVrbDaiU3jh2zVZyJNu8Lr6TtSb9PrFN8
X-Received: by 2002:a37:a407:: with SMTP id n7mr27806250qke.46.1550764664718;
        Thu, 21 Feb 2019 07:57:44 -0800 (PST)
Return-Path: <MyFakeYahooAccount@yahoo.com>
Received: from sonic312-20.consmr.mail.bf2.yahoo.com (sonic312-20.consmr.mail.bf2.yahoo.com. [74.6.128.82])
        by mx.google.com with ESMTPS id p1si2743950qtp.365.2019.02.21.07.57.44
        for <MyFakeGMailAccount@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 21 Feb 2019 07:57:44 -0800 (PST)
Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.bf2.yahoo.com with HTTP; Thu, 21 Feb 2019 15:57:44 +0000
Date: Thu, 21 Feb 2019 15:57:40 +0000 (UTC)
From: RobiBue <MyFakeYahooAccount@yahoo.com>
Message-ID: <1883948137.3003972.1550764660065@mail.yahoo.com>
Subject: test with Bcc: address
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_Part_3003971_1789674874.1550764660064"
References: <1883948137.3003972.1550764660065.ref@mail.yahoo.com>
X-Mailer: WebService/1.1.13123 YMailNorrin Mozilla/5.0 (Windows NT 6.1; WOW64; rv:67.0) Gecko/20100101 Firefox/67.0
Content-Length: 505

As you can see, the Bcc: seen in the outgoing message was dropped, as it is not expected nor needed.

Next, I reply (with Bcc:) to the above message (keep in mind that I am only showing the headers to keep it concise:

outgoing reply to test message from GMail:
------------------------------------------

MIME-Version: 1.0
Received: by 2002:a0c:acd6:0:0:0:0:0 with HTTP; Thu, 21 Feb 2019 08:00:49 -0800 (PST)
Bcc: RobiBue <MyFakeYahooAccount@yahoo.com>
In-Reply-To: <1883948137.3003972.1550764660065@mail.yahoo.com>
References: <1883948137.3003972.1550764660065.ref@mail.yahoo.com> <1883948137.3003972.1550764660065@mail.yahoo.com>
Date: Thu, 21 Feb 2019 10:00:49 -0600
Delivered-To: MyFakeGMailAccount@gmail.com
Message-ID: <CABr3MG+L8J5N2tp5bWir+PyrYOeo5aJs=XsfDeKqROayovyTRQ@mail.gmail.com>
Subject: Re: test with Bcc: address
From: RobiBue <MyFakeGMailAccount@gmail.com>
To: undisclosed-recipients:;
Content-Type: text/plain; charset="UTF-8"

and Yahoo! receives the following:

incoming reply from GMail in Yahoo!:
------------------------------------

X-Apparently-To: MyFakeYahooAccount@yahoo.com; Thu, 21 Feb 2019 16:00:51 +0000
Return-Path: <MyFakeGMailAccount@gmail.com>
Received-SPF: pass (domain of gmail.com designates 209.85.160.178 as permitted sender)
X-Originating-IP: [209.85.160.178]
Authentication-Results: mta4039.mail.bf1.yahoo.com 
 header.i=@gmail.com; header.s=20161025; dkim=pass (ok)
Received: from 127.0.0.1  (EHLO mail-qt1-f178.google.com) (209.85.160.178)
  by mta4039.mail.bf1.yahoo.com with SMTPS; Thu, 21 Feb 2019 16:00:50 +0000
Received: by mail-qt1-f178.google.com with SMTP id z25so10287996qti.13
        for <MyFakeYahooAccount@yahoo.com>; Thu, 21 Feb 2019 08:00:50 -0800 (PST)
X-Received: by 2002:ac8:2bc1:: with SMTP id n1mr31583604qtn.176.1550764849978;
 Thu, 21 Feb 2019 08:00:49 -0800 (PST)
MIME-Version: 1.0
Received: by 2002:a0c:acd6:0:0:0:0:0 with HTTP; Thu, 21 Feb 2019 08:00:49
 -0800 (PST)
In-Reply-To: <1883948137.3003972.1550764660065@mail.yahoo.com>
References: <1883948137.3003972.1550764660065.ref@mail.yahoo.com> <1883948137.3003972.1550764660065@mail.yahoo.com>
From: RobiBue <MyFakeGMailAccount@gmail.com>
Date: Thu, 21 Feb 2019 10:00:49 -0600
Message-ID: <CABr3MG+L8J5N2tp5bWir+PyrYOeo5aJs=XsfDeKqROayovyTRQ@mail.gmail.com>
Subject: Re: test with Bcc: address
To: undisclosed-recipients:;
Content-Type: text/plain; charset="UTF-8"
Bcc: MyFakeYahooAccount@yahoo.com
Content-Length: 82

As you can see, Google leaves the Bcc: in the headers (although I didn't test it, but I believe if I had more bcc addresses there, only the end recipient getting the message would be listed alone)

I do note, that if the Content-Length: header wouldn't have been added, the Bcc: line would have been at the bottom and with it, SC would have trimmed the body off, but it's there, and SC parses it for me.

To recap: if you have a Bcc: cutting off the body in SC, remove the Bcc: as it's not needed anyway and nobody expects it to be there anyway.

 

[MIG]

3 hours ago, RobiBue said:

As I mentioned in my previous message, I did a test between Yahoo! and GMail with Bcc: header test.

Here the "munged" but true header results:

The above outgoing message (in the sent folder) shows that the Bcc: was there when it left the Yahoo! mailbox.

Next comes the GMail inbox part:

As you can see, the Bcc: seen in the outgoing message was dropped, as it is not expected nor needed.

Next, I reply (with Bcc:) to the above message (keep in mind that I am only showing the headers to keep it concise:

and Yahoo! receives the following:

As you can see, Google leaves the Bcc: in the headers (although I didn't test it, but I believe if I had more bcc addresses there, only the end recipient getting the message would be listed alone)

I do note, that if the Content-Length: header wouldn't have been added, the Bcc: line would have been at the bottom and with it, SC would have trimmed the body off, but it's there, and SC parses it for me.

To recap: if you have a Bcc: cutting off the body in SC, remove the Bcc: as it's not needed anyway and nobody expects it to be there anyway.

 

Hey RobiBue & Lking,

Thanks for pitching in and your considered reviews. I did remove the bcc last night & re-parse, however, my parsing was/is hitting a wall with 2 errors, not the same as euphorique's. And, I'm 99% sure Euphorique has also tried [remove bcc] also without successful result...

My parsing errors are: "Mailhost configuration problem, identified internal IP as source" & "No source IP address found, cannot proceed", it's been a while since I set up my hosts, I'm pretty sure these errors are easily fixed, just trying to sort now, really don't want to remove one of my working hosts if it's going to break my setup when I revert back; anyway, once fixed,  I'll post back. 

In the interim would either of you like the raw source data? If yes may I pm to you? 

Euphorique, if you're back, welcome!

Personally, given the options I think I'd pitch for grass🦗mahopper, the Donner history gives me chills.

Cheers!

 

[euphorique]

5 hours ago, MIG said:

To recap: if you have a Bcc: cutting off the body in SC, remove the Bcc: as it's not needed anyway and nobody expects it to be there anyway.

I completely agree on the purpose of Bcc: header, and acknowledge that it can be handled differently in various mail agents, correct or otherwise.

Just consider the case with Bcc: header not being the last one in the header block:

Received: from mbkd0231.ocn.ad.jp (mbkd0231.ocn.ad.jp. [153.149.233.32])
        by mx.google.com with ESMTP id e96si22718016plb.123.2019.02.21.05.54.06;
        Thu, 25 Feb 2019 05:54:07 -0800 (PST)
Bcc: spammer@spam.com
From: spammer@spam.com

Hey! There is a blank line between the headers and the body!

 

SC parses Bcc: header correctly. The parser is aware that Bcc: header can rightfully contain the real address, and crosses the address out:

Skip to Reports

Received: from mbkd0231.ocn.ad.jp (mbkd0231.ocn.ad.jp. [153.149.233.32])
        by mx.google.com with ESMTP id e96si22718016plb.123.2019.02.21.05.54.06;
        Thu, 25 Feb 2019 05:54:07 -0800 (PST)
Bcc: x
From: spammer@spam.com

View entire message

 

When Bcc: header is the last one in the header block, the parser chops off the entire message body.

 

I can only speculate on the implementation of the parser, but in my imagination the error is something rather simple and obvious (given the number of 5-line test cases submitted here), and does not require a terrific effort to fix.

And yes, for now I remove the Bcc: header if it breaks the parser.

 

Cheers, it's Friday!

[petzl]

From: spammer[]spam.cxm

Hey! There is a blank line between the headers and the body!

Needs TWO blank lines
Spamware often does not separate headers from body and if it has 2000+ spam victim email address to "X" out
in a visible BCC field it will have a hernia!

[MIG]

10 hours ago, petzl said:

Needs TWO blank lines
Spamware often does not separate headers from body and if it has 2000+ spam victim email address to "X" out
in a visible BCC field it will have a hernia!

  Tried that, the only one with a hernia is grass🦗hopper,

[petzl]

16 hours ago, MIG said:

  Tried that, the only one with a hernia is grass🦗hopper,

You need to also remove the "gillion or so" BCC addresses, replace with a X.

[MIG]

On 2/23/2019 at 12:50 AM, euphorique said:

I completely agree on the purpose of Bcc: header, and acknowledge that it can be handled differently in various mail agents, correct or otherwise.

Just consider the case with Bcc: header not being the last one in the header block:

Received: from mbkd0231.ocn.ad.jp (mbkd0231.ocn.ad.jp. [153.149.233.32])
        by mx.google.com with ESMTP id e96si22718016plb.123.2019.02.21.05.54.06;
        Thu, 25 Feb 2019 05:54:07 -0800 (PST)
Bcc: spammer@spam.com
From: spammer@spam.com

Hey! There is a blank line between the headers and the body!

 

SC parses Bcc: header correctly. The parser is aware that Bcc: header can rightfully contain the real address, and crosses the address out:

Skip to Reports

Received: from mbkd0231.ocn.ad.jp (mbkd0231.ocn.ad.jp. [153.149.233.32])
        by mx.google.com with ESMTP id e96si22718016plb.123.2019.02.21.05.54.06;
        Thu, 25 Feb 2019 05:54:07 -0800 (PST)
Bcc: x
From: spammer@spam.com

View entire message

 

When Bcc: header is the last one in the header block, the parser chops off the entire message body.

 

I can only speculate on the implementation of the parser, but in my imagination the error is something rather simple and obvious (given the number of 5-line test cases submitted here), and does not require a terrific effort to fix.

And yes, for now I remove the Bcc: header if it breaks the parser.

 

Cheers, it's Friday!

Hey euphorique, 

RobiBue worked on this:

Method 1: removed bcc: since it's the last line of the header.

Result: 209.85.220.65 : network-abuseATgoogleDOTcom

Method 2: moved bcc to second last line in header.

Result: 209.85.220.65 : network-abuseATgoogleDOTcom

RobiBue added a comment ( "I don't have Mailhost configuration set up. That might make a difference") which got grass🦗hopper thinking...

> I set up a dummy SC account without MailHosts, made the same tweaks & voila:

Method 1 (no bcc) 

Result: 209.85.220.65 : network-abuseATgoogleDOTcom

Method 2 (bcc changed to 2nd last header line)

Result: 209.85.220.65 : network-abuseATgoogleDOTcom

This msg:  "No body text provided, check format of submission. spam must have body text" is gorne😊

When you're back euphorique, please chk parsing spam thru an account without MailHosts, let us know the outcome please?

Mega  grass🦗hopper  🙏thanks🙏to RobiBue.

Cheers!

 

 

 

[klappa]

On 2/22/2019 at 8:37 AM, MIG said:

Hey RobiBue & Lking,

Thanks for pitching in and your considered reviews. I did remove the bcc last night & re-parse, however, my parsing was/is hitting a wall with 2 errors, not the same as euphorique's. And, I'm 99% sure Euphorique has also tried [remove bcc] also without successful result...

My parsing errors are: "Mailhost configuration problem, identified internal IP as source" & "No source IP address found, cannot proceed", it's been a while since I set up my hosts, I'm pretty sure these errors are easily fixed, just trying to sort now, really don't want to remove one of my working hosts if it's going to break my setup when I revert back; anyway, once fixed,  I'll post back. 

In the interim would either of you like the raw source data? If yes may I pm to you? 

Euphorique, if you're back, welcome!

Personally, given the options I think I'd pitch for grass🦗mahopper, the Donner history gives me chills.

Cheers!

 

What's with all the grassmahopper gifs in your text?

[Lking]

4 hours ago, klappa said:

What's with all the grassmahopper gifs in your text?

guilty 😁

[lisati]

8 hours ago, klappa said:

What's with all the grassmahopper gifs in your text?

😁