Personal Whitelist Problem

[Guest SkydiveMike]

Your "problem reporting" / "request for help" isn't quite a correct depiction of the situation at this point.

Yes, it is.

maybe it's been read and blown off, maybe it's been put on a back burner, maybe everyone is swamped with other things, and it's time for the dog to go out which also means it's time to do some more snow shoveling ....

22585[/snapback]

And maybe, just maybe, it has been blow off because the discussion of the syptoms, facts, my evidenciay posting, etc. is in an off topic lounge forum.

[jefft]

http://www.spamcop.net/sc?id=z709298221za9...236b7ccdc0d33fz

22361[/snapback]

The filtering was being done on the Return-Path header. However, it appears that pobox.com is now totally munging the Return-Path for all email they forward. I believe this is SRS, an ugly hack which is necessary to make SPF work. Since the SPF inventor is a pobox.com guy (I think) it's not surprising they're the first to do this.

See: SRS Info for more details.

The bottom line is that this totally breaks how we're doing whitelists, at least for the Return-Path header. If you can't whitelist on From: then you're out of luck for now. There isn't a quick or easy fix.

JT

[Wazoo]

Thanks. Definitive data and response much appreciated.

Sorry that the words of others weren't sufficient ....

More data found at http://spf.pobox.com/srs.html

Actual "problem" found at http://www.libsrs2.org/srs/srs.pdf

[Jeff G.]

My, what a long Topic! However, I believe I have something to add.

WRT Mike's problem with Yahoo! Groups emails, I agree that he could stop sending them through pobox.com and/or ask pobox.com to stop muckng with the Return-Path. He could also try whitelisting "mikes.mailinglists=pobox.com[at]bounce2.pobox.com" without quotes, but that might be leaving himself too open for forgery.

WRT Mike's problem with ISP replies not being whitelisted, contacting SpamCop Parsing and Reporting System Admins is the way to go. I don't usually get replies from ISPs, and the ones I do get come from ISPs so responsible that they don't let their servers get listed on the SCBL, so I can't reproduce the problem at present. It would be nice if there were a low-use autoresponder on an SCBL-listed server that we could use for testing. When the SpamCop Parsing and Reporting System's Server(s) at reports.spamcop.net process incoming email, they: WRONGLY CREATE THE ORIGINAL "Return-Path" HEADER LINE (wrongly because they are NOT the final SMTP transport system); determine which user to send it to; prepend a "X-SpamCop-Reply-Ids" Line containing the Report ID that was in the "RCPT TO" transaction in the SMTP session to the Header; prepend a "X-SpamCop-Return-Path" Line containing the original "Return-Path" contents to the Header; and pass the message on one of the vmx boxes using a "MAIL FROM" of "spamcop at devnull.spamcop.net" (munged slightly for the web). The ISPs doing the replying have nothing to do with this process, and the changing of the Return-Path is presumably done to help the SpamCop Admins keep a tighter rein and to prevent compliant bounces from revealing reporters' secret email addresses. Note that all Tracking URLs

http://www.spamcop.net/sc?id=z709451940z9d...f9853c54d6c5dez and http://www.spamcop.net/sc?id=z709641962z6e...9b07e8599b2b21z and

http://www.spamcop.net/sc?id=z709795403z74...1e9b2db6ec6025z and

http://www.spamcop.net/sc?id=z709795404z03...78bc362d577ec9z show emails with not

one but TWO Return-Path Header Lines each, the first/lower created by SpamCop Parsing and Reporting System Server sc-app4.eq.ironport.com. In each case, the first/lower one should not have been created by sc-app4.eq.ironport.com, but it was in fact created. This appears to be an violation of Section 4.3.1 of Internet Standard #11 and RFC 822 "Standard for the format of ARPA Internet text messages" ("RETURN-PATH ... This field is added by the final transport system that delivers the message to its recipient"), Section 4.4 of RFC 2821 "Simple Mail Transfer Protocol" ('When the delivery SMTP server makes the "final delivery" of a message, it inserts a return-path line at the beginning of the mail data' ... 'Here, final delivery means the message has left the SMTP environment'), and Section 3.6.7 of RFC 2822 "Internet Message Format" ('an optional "Return-Path:" field'), perpetrated within the SpamCop Parsing and Reporting System, and I am not one bit surprised that the whitelists would be taking a random Return-Path Header Line as being the one true one, and ignoring the top one. The fact that there are other entries on Mike's Personal Whitelist should have no bearing whatsoever on the SpamCop Email System's ability to correctly filter his mail with a "Return-Path" of "spamcop at devnull.spamcop.net" given that he has "spamcop.net" on his Personal Whitelist, and there is no way '"Block Everything" hifing[sic] under the blacked-out stuff' is happening. Since I believe this problem to be one in which SpamCop Parsing and Reporting System Server sc-app4.eq.ironport.com is messing up and SpamCop Email System Servers are correctly relying on published SMTP RFCs, I'd say the problem is in the SpamCop Parsing and Reporting System's bailywick, and should be reported via address service at admin.spamcop.net and discussed in the SpamCop Help Forum, rather than in the Lounge. I believe the double "Return-Path" problem is probably there for free and fuel-based reporters, too, so even if JT was able to compensate on Mike's behalf, that would still leave free and fuel-based reporters with very strict mailservers in the lurch.

WRT Mike's problem with Wazoo's consolidation of his Topics into the Lounge, in this case this is not as big a problem as it appears, because there is still a pointer to each moved Topic where it moved from (that's how I found it before finishing my current reading of the SpamCop Email Forum). I will admit that the SpamCop Email Forum is the second Forum I read, after the Announcements Forum, and that the Lounge is one of the last Forums that I read. The action was justified because Mike was not cooperating in having Yahoo! Groups deliver his email directly to SpamCop's servers, bypassing pobox.com's servers, but now that he has dropped the Yahoo! Groups issue, "SpamCop Help" is the way to go.

[agsteele]

I didn't use the word conspiracy, you did. Please do not put words in my mouth. A moderator did "take action" and move my problem report. At least one other user has expressed that he/she would not have moved the thread. While all forums get attention, is it possible, even a little bit, that the "real problem" forums get more or more frequent attention than the "lounge?" This is the gist of my point (not some conspiracy like you allude) and I think it is very fair.

I have sent email; Is it possible, even a little bit, that they are so busy that they "accept" on faith that since the moderator has moved the thread to the lounge it is not "really" an issue and therefore have not even looked into it? I think so.

22514[/snapback]

Yes, I did use the word 'conspiracy' since my reading of what you were saying seemed to suggest that was what you perceived to be the case. Thanks for the clarification.

However, you have no evidence that your problem, moved to the Lounge, has received any less attention than if it had been left in its original location. As you say: "I think so".

What others are saying is that our experience runs counter to this and confirming that the way to get employee help is via Email rather than the forums except for those occasions when an employee happens to drop by.

Is it possible that they will give less attentiont to a lounge topic than one in one of the other forums? I guess it could, but 'it is possible, even a little bit' that they might also only visit because they enjoy the debate in the Lounge and that could be their favourite starting point. Who can say?

I can say that my experience is that help generally gets provided in whatever forum a message is placed.

I now see that JT has posted an answer in this thread in the Lounge which confirms his visit here.

Andrew

[Guest SkydiveMike]

Thanks.  Definitive data and response much appreciated.

Sorry that the words of others weren't sufficient ....

More data found at http://spf.pobox.com/srs.html

Actual "problem" found at http://www.libsrs2.org/srs/srs.pdf

22592[/snapback]

It is not definative because it does not address the abuse[at]isp problem ...

[Guest SkydiveMike]

WRT Mike's problem with Wazoo's consolidation of his Topics into the Lounge, in this case this is not as big a problem as it appears, because there is still a pointer to each moved Topic where it moved from (that's how I found it before finishing my current reading of the SpamCop Email Forum).  I will admit that the SpamCop Email Forum is the second Forum I read, after the Announcements Forum, and that the Lounge is one of the last Forums that I read.  The action was justified because Mike was not cooperating in having Yahoo! Groups deliver his email directly to SpamCop's servers, bypassing pobox.com's servers, but now that he has dropped the Yahoo! Groups issue, "SpamCop Help" is the way to go.

22594[/snapback]

Thank you for the answers WRT to incoming ISP abuse responses -- some real help is finally and muchly appreciated.

To address your paragraph above however -- my initial problem report was a whitelist problem not a yahoogroups problem. The confusion and merging between whitelist-problem and yahoogroups-symptom was Wazoo's confusion -- not mine.

In particular I posted This Message as a brand new forum topic in the support forum -- Wazoo merged it into the existing thread, which by that time was focusing on yahoogroups.

The thread moving / merging / hiding in the lounge (and yes I can use the word hiding when helpful people such as yourself admit they read the lounge last) was the root cause for the length and confusion of this thread.

My abuse[at]isp issue could have been solved with 2-3 posts in the support forum had it been allowed to stay there and someone helpful such as yourself posted a helpful reply as you did. There were 10-20 messages just trying to clarify that there were, indeed, two issues and none of these would have been necessary if Wazoo had left well enough alone.

[Guest SkydiveMike]

Actual "problem" found at http://www.libsrs2.org/srs/srs.pdf

22592[/snapback]

I am not playing word games here -- please drop the pent up emotion when you read the following sentance.

SPF is not the actual problem; the actual problem is that pobox supports SPF and spamcop does not support SPF

There is a difference; it is not a word game; it is a valid and important distinction.

[Miss Betsy]

To address your paragraph above however -- my initial problem report was a whitelist problem not a yahoogroups problem. The confusion and merging between whitelist-problem and yahoogroups-symptom was Wazoo's confusion -- not mine.

I don't like to be picky, but you were the one insisting that your problem might have something to do with the problems others were experiencing with yahoo when everyone else was telling you that yours was a whitelisting problem peculiar to you.

I do hope that you have solved your problems satisfactorily now. It is very frustrating when there is a seemingly inexplicable problem and there is certainly nothing wrong with exploring different avenues to solve it.

Miss Betsy

[Guest SkydiveMike]

When the SpamCop Parsing and Reporting System's Server(s) at reports.spamcop.net process incoming email, they: WRONGLY CREATE THE ORIGINAL "Return-Path" HEADER LINE (wrongly because they are NOT the final SMTP transport system); determine which user to send it to; prepend a "X-SpamCop-Reply-Ids" Line containing the Report ID that was in the "RCPT TO" transaction in the SMTP session to the Header; prepend a "X-SpamCop-Return-Path" Line containing the original "Return-Path" contents to the Header; and pass the message on one of the vmx boxes using a "MAIL FROM" of "spamcop at devnull.spamcop.net" (munged slightly for the web).  The ISPs doing the replying have nothing to do with this process, and the changing of the Return-Path is presumably done to help the SpamCop Admins keep a tighter rein and to prevent compliant bounces from revealing reporters' secret email addresses.

Ok, so the spamcop system is doing something to the return-path header to assist in the maintenance of the report submitters identity -- this is well and good and expected.

Note that all Tracking URLs <snip> show emails with not one but TWO Return-Path Header Lines each, the first/lower created by SpamCop Parsing and Reporting System Server sc-app4.eq.ironport.com.

So the ISP reports that get placed in my held mail folder are malformed from the beginning and spamcop doesn't handle malformed incoming mail -- again, this is well and good and accepted. Thank you, thank you, thank you for actually finding this incoming ISP reports problem buried on page 9 of a thread in the lounge.

I apologize profusely for the extra work that Wazoo put you through to to find this report, separate it conceptually from the yahoogroup sympton, and respond to it.

The fact that there are other entries on Mike's Personal Whitelist should have no bearing whatsoever on the SpamCop Email System's ability to correctly filter his mail with a "Return-Path" of "spamcop at devnull.spamcop.net" given that he has "spamcop.net" on his Personal Whitelist, and there is no way '"Block Everything" hifing[sic] under the blacked-out stuff' is happening.

Thank you for your support in this -- I found it strange that Wazoo asked twice for the email addresses of my friends and relatives.

I'd say the problem is in the SpamCop Parsing and Reporting System's bailywick, and should be reported via address service at admin.spamcop.net and discussed in the SpamCop Help Forum, rather than in the Lounge.  I believe the double "Return-Path" problem is probably there for free and fuel-based reporters, too, so even if JT was able to compensate on Mike's behalf, that would still leave free and fuel-based reporters with very strict mailservers in the lurch.

Wow -- yet another extremely helpful person that concurs that this is a spamcop problem and that this discussion does not belong in the lounge. I have been beggin and pleading both privately and publicall for it to moved out of the lounge. Thank you, thank you, thank you !!!!!

[Guest SkydiveMike]

Yes, I did use the word 'conspiracy' since my reading of what you were saying seemed to suggest that was what you perceived to be the case.  Thanks for the clarification.

Why should a paying customer (me in this case) asking for support and service need to explain that he/she is not a conspiracy nut? Especially when he/she never used the word conspiracy?

However, you have no evidence that your problem, moved to the Lounge, has received any less attention than if it had been left in its original location.  As you say: "I think so".

What others are saying is that our experience runs counter to this and confirming that the way to get employee help is via Email rather than the forums except for those occasions when an employee happens to drop by.

Is it possible that they will give less attentiont to a lounge topic than one in one of the other forums?  I guess it could, but 'it is possible, even a little bit' that they might also only visit because they enjoy the debate in the Lounge and that could be their favourite starting point.  Who can say?

Read the entire thread; there are two other (i.e. not me) forum members expressing a belief that this discussion did not belong in the lounge and at least one person who admints that he/she reads the lounge last.

[Guest SkydiveMike]

I don't like to be picky, but you were the one insisting that your problem might have something to do with the problems others were experiencing with yahoo when everyone else was telling you that yours was a whitelisting problem peculiar to you.

22607[/snapback]

I initially reported the problem as a whitelist problem (read my subject line)

Wazoo pointed me to the other yahoogroups problems that were coincident in time with my problem. I stated that there might be a relationship -- I never insisted there was a relationship.

I posted a new topic regarding incoming isp responses, as a new topic in the support forum. Wazoo moved that into the topic that was (by that point) focused on yahoogroups.

[Wazoo]

Well, the suggested views of moving this somewhere else were seen, evaluated, then blowm out of contention by thi slast series of rants, mis-statements, etc. Yet again, this type and content of conversation does not easily fit into a "Help" scenario.

Just a specific: "Wazoo asked twice for the email addresses of my friends and relatives."

I'll ask you to please document just when and where that happened.

I'm not holding my breath for a response, based on the history of questions asked, answers provided. Though there has finally been some progress made on your behalf, the rant scenario of this entire conversation has made its impact too severe. There is dialog going with JeffG's latest findings, but that's another dance. If you'd followed links to the provided resources on SPF/SRS, you'd have found that implemeting these has this strange result of "breaking e-mail forwarding" in a host of situations .. mentioned by the originator and other developers as a major issue in implementing this stuff at the present time, calling for new RFCs to be developed and implemented (and I add world-wide to that)

[StevenUnderwood]

WRT Mike's problem with ISP replies not being whitelisted, contacting SpamCop Parsing and Reporting System Admins is the way to go.  I don't usually get replies from ISPs, and the ones I do get come from ISPs so responsible that they don't let their servers get listed on the SCBL, so I can't reproduce the problem at present.

...

I am not one bit surprised that the whitelists would be taking a random Return-Path Header Line as being the one true one, and ignoring the top one

....

Since I believe this problem to be one in which SpamCop Parsing and Reporting System Server sc-app4.eq.ironport.com is messing up and SpamCop Email System Servers are correctly relying on published SMTP RFCs, I'd say the problem is in the SpamCop Parsing and Reporting System's bailywick, and should be reported via address service at admin.spamcop.net and discussed in the SpamCop Help Forum, rather than in the Lounge.  I believe the double "Return-Path" problem is probably there for free and fuel-based reporters, too, so even if JT was able to compensate on Mike's behalf, that would still leave free and fuel-based reporters with very strict mailservers in the lurch.

22594[/snapback]

Jeff:

While this may be the problem, since this problem started, I have changed my settings to receive ALL replies and whitelisted spamcop.net (to match Mikes settings) and have not seen one instance where the whitelist did not catch the message. I did post 2 tracking URL's somewhere in this thread. As you state, none of the 12 messages I have received has been on a blocklist, so that MAY play into it as well.

[Wazoo]

I initially reported the problem as a whitelist problem (read my subject line)

Wazoo pointed me to the other yahoogroups problems that were coincident in time with my problem. I stated that there might be a relationship -- I never insisted there was a relationship.

I'm sorry ... your first post repeated below, as you obviously haven't read it since you p0sted it;

Over the last few days I have been seeing problems with my personal whitelist.

I have

returns.groups.yahoo.com

groups.yahoo.com

in my whitelist (and have had for a while -- and it has worked in the past).

I am now (past few days) getting all of my yahoo groups email in my held mail; not my inbox. I tried re-adding the addresses above to my whitelist and nothing has changed.

Any thoughts?

You may not have "insisted" .. but it's sure clear to me how this thing got started.

I posted a new topic regarding incoming isp responses, as a new topic in the support forum. Wazoo moved that into the topic that was (by that point) focused on yahoogroups.

Please try re-reading the above and try again. And as stated in most of those Move/Merge actions, your new posts were seen as just an expansion upon that first post, starting with a YahooGroups issue, then expanding it out to one other item, then stating that it didnt work at all, then changing it to that it worked sometimes ... thus the "need" for it all to stay in one place so folks would be wasting time trying to handle different "versions" and "specific" of your issue. Spending your time working out your rant mode, yet not getting involved with working the issue is why this 'conversation' is still in the Lounge.

[Miss Betsy]

I stated that there might be a relationship -- I never insisted there was a relationship.

From one of SkydiverMike's earlier posts

(*) Coincidently, the beginning January timeframe also shows many other users reporting problems with yahoogroups -- though not necessarily verified and isolated to the spamcop whitelist -- intuition says these are probably related and it certainly is a good point for the developers to start with their root cause analysis search.

I think there was another one that ended in "I think so" but I am not going to search for it. However, since this was several pages back, I admit that he has not 'insisted' (or at least didn't for more than one or two posts which is not unreasonable).

Miss Betsy

[Wazoo]

I certainly won't argue with Miss Betsy .... However, the referenced intial suggestion to check out the "other" YahooGroups thing was to note that those complaints were from folks using "other ISPs" .. nothing to do with the SpamCop E-mail account server .... the next "step" was the posting of his whitelist issue within that Topic, which everyone has agreed thus far, had no relationship or bearing on the issue of SkydiveMike's white-list issue. (That whole part of "that" conversation is now contained within "this" conversation.)

Then, StevenUnderwood pointed out the Reply-To" line condition. It was a long time coming in that there was any Forwarding involved. But in the interim, the only "help" provided by the user with a problem was the rant that "it doesn't work" ... Eventually, DavidT did some more work, came up with the same results, yet not much more than the continued "it doesn't work" provided ...

Then it went to the "move on" thing, let's focus on yet another item that "proves" it doesn't work. Again, StevenUnderwood attempted to point some details out. Response was more of the "it doesn't work" song.

JT finally got/took the time to take a look, posted the results .. solution is hinted at ... response? More of the "it doesn't work" song.

At present, JeffG has made notifies of his findings and I can tell you that this is being looked at. JT pointed out the issues of the pobox configuration. Not known is what SkydiveMike has attempted to do with any of the results at this point. And once again, if there were another batch of complaints from other users making the same complaint (that the white-list doesn't work, well, maybe, well sometimes, well I can't tell...) I'd really be a lot more enthused about things. However, ....

(not referenced in the above were the additional postings of the YahooGroups issue into even more discussions elsewhere that were also moved/merged into this single Topic .. so, yes one could say "didn't insist" .. but .... )

[StevenUnderwood]

And as a reply to my last post where I had not had any replies from blocklisted servers, I have now had a couple from the same ISP in Brazil which were blocklisted and the whitelist spamcop.net worked as expected.

I will post tracking URL's in this message when I receive them back.

And here they are:

http://www.spamcop.net/sc?id=z710085670zf9...d21fac382dd23bz

http://www.spamcop.net/sc?id=z710085672zd5...5a2203c9b8bb1az

You will notice both of these have:

X-SpamCop-Disposition: Blocked brazil.blackholes.us

X-SpamCop-Whitelisted: spamcop.net

What I find interesting is that the second parse has the "You appear to be reporting a reply from an ISP as spam!" warning but the first does not, yet they both are re: the same spam.

[jefft]

Now, for some reason, incoming responses to spam complaints are not being whitelisted by the spamcop system and being placed in Held Mail. Similar to the yahoogroups whitelist problem I have released one message and then cancelled the reports to generate a tracking URL:

http://www.spamcop.net/sc?id=z709451940z9d...f9853c54d6c5dez

Any clue on what has gone wrong with spamcop now?

22391[/snapback]

You don't have spamcop.net on your whitelist. What exactly were you thinking would whitelist this message?

JT

p.s. If you do have spamcop.net on your whitelist, you need to email me your username because the user I looked at does not.

[Wazoo]

You don't have spamcop.net on your whitelist. What exactly were you thinking would whitelist this message?

JT

p.s. If you do have spamcop.net on your whitelist, you need to email me your username because the user I looked at does not.

Geeeze .. thus can't be true. Lemme see, SkydiveMike e-mailed you from his SpamCop account as he states in Post #111;

I sent if from my spamcop account via HORDE so my account information is present for JT.

Post #61 clearly states;

I do have spamcop.net in my white list which should match devnull.spamcop.net and/or spamcop[at]devnull.spamcop.net as per the documentation. I do not understand how any of these emails end up in my held mail.

Things just keep getting weirder and weirder ....

[Wazoo]

A bit of input from Don .. maybe something to be added to the White-list / Filter FAQ?

The SpamCop Email Filters look in the user's personal whitelist for matches on these fields:

# Envelope Sender aka Return Path

# From:

# Sender:

The filters will see *any* Return-Path header even if there is more than one.

Users wishing to prevent ISP replies coming from blocked servers from being

diverted to Held Mail should put  devnull.spamcop.net in their whitelist.

Putting just  spamcop.net  on the list will work, but it will also allow spam

from forged SpamCop addresses to pass.

-Don-

[Guest SkydiveMike]

Just a specific: "Wazoo asked twice for the email addresses of my friends and relatives."

I'll ask you to please document just when and where that happened.

You wanted to see what was "blacked out" on the screen shots of my whitelist. You asked twice.

Though there has finally been some progress made on your behalf, the rant scenario of this entire conversation has made its impact too severe.

Is that even a valid sentance?

There is dialog going with JeffG's latest findings, but that's another dance.

Since when is a customer looking for help a "dance"?

If you'd followed links to the provided resources on SPF/SRS, you'd have found that implemeting these has this strange result of "breaking e-mail forwarding" in a host of situations

I did follow the links, I understand the issue with SPF/SRS -- what you don't understand is that email from abuse[at]isp to munged-me[at]spamcop does not go through pobox, does not have SPF/SRS stuff in it, has a return path containing spamcop.net, I have spamcop.net in my whitelist, and the mail still ends up in held mail -- ergo the whitelist doesn't work -- back to my original problem (aka subject == personal whitelist problem).

I didn't see any technical questions or requests for information in your post so I can't answer any. If you look through the topic carefully you will see that I have answered every question and provided technical details (tracking URL's) in response to every request. Most of those requests came from other forum users, not you.

[Guest SkydiveMike]

Jeff:

While this may be the problem, since this problem started, I have changed my settings to receive ALL replies and whitelisted spamcop.net (to match Mikes settings) and have not seen one instance where the whitelist did not catch the message.  I did post 2 tracking URL's somewhere in this thread.  As you state, none of the 12 messages I have received has been on a blocklist, so that MAY play into it as well.

22636[/snapback]

Here are two more instances from today. Both of these have "Return-Path: <spamcop[at]devnull.spamcop.net>" as the first header.

http://www.spamcop.net/sc?id=z710127556zba...a83240ad087db7z

http://www.spamcop.net/sc?id=z710127558z16...741466f082f607z

Wazoo -- does this count as answering questions and providing information or a rant? Just curious.

[Guest SkydiveMike]

You don't have spamcop.net on your whitelist. What exactly were you thinking would whitelist this message?

p.s. If you do have spamcop.net on your whitelist, you need to email me your username because the user I looked at does not.

22668[/snapback]

I did have spamcop.net on my whitelist and didn't ever delete it. My spamcop mail is mike.mclean[at]spamcop.net

[Guest SkydiveMike]

Things just keep getting weirder and weirder ....

22674[/snapback]

Gee, if something is happening that is corrupting / modifying my whitelist then maybe, just maybe, the subject line of my very first message (Personal Whitelist problem) is an accurate description of the problem.