[Resolved]Â Got back both success and failure notices

[Vanguard 0]

I went through the Mailhosts procedure mentioned at mhconf.<code removed>[at]cmds.spamcop.net by clicking on the bottom link. I got the confirmation e-mails (2 of them because 2 gateways were listed for my Comcast provider). One option was to forward that e-mail as an attachment to a specific SpamCop address.

Outlook has the option to forward as an attachment and that is how I configure it to behave. Forwarding inline loses all the headers for the original e-mail but they are there when sent as an attachment. When forwarding e-mails, I want the recipient to get the same exact message that I got, including the headers. A very short time later, I got 2 more e-mails for each test (for a total of 4 e-mails for the 2 confirmations which I forwarded as attachments). Basically, for each "SpamCop account configuration email" message that I forwarded back as an attachment, I got 2 replies:

"SpamCop account configuration: error"

"SpamCop account configuration: success"

Because these replies were sent out almost at the same time, they might appear in the order shown above or in reverse order. So, at this point, and because there is no way to check on the status of my account regarding this mailhost parsing setup, I don't know if the setup was successful or failed.

When attaching an e-mail as an attachment when forwarding it, it gets attached as a .msg file. Maybe SpamCop doesn't know how to parse and interpret .msg files. If so, I could use the web form where I paste the headers and body. However, at this point, I don't know if I should do that since I don't if it trying requalify a success would result in a problem.

How do I check if the mailhost check was a success (since I got both a success and failure message)? Should I try using the web page link in the first e-mail ("SpamCop account configuration email") to try again?

[Vanguard 0]

Okay, I tried the web form reporting mechanism. From the first SpamCop e-mail titled "spam account configuration email", I clicked on the link to go to the web page. I copied the headers and the body in the appropriate textboxes and submitted. I got back:

Confirmation codes do not match:

From recipient address: jgCkjGszP8sWNCYT                                           
From header:            jgCkjGszP8sWNCYT
From body:              jgCkjGszP8sWNCYT

Huh? They look the same to me. Do I trash those e-mails and go back to the Mailhosts web page and click on the link to get another set of test e-mails from SpamCop?

[Wazoo 0]

Problem #1 ... I believe you have exposed some code here.

Problem #2 ... Outlook does not play well with SpamCop, documented all over the place ..

Problem #3 ... The problem you seem to be describing (ignoring the above) is an issue of whitespace. One of those codes more than likely is followed by a bunch of 'space' characters (?) that you're not seeing, but because they are being supplied in your response, the "codes don't match" (see http://forum.spamcop.net/forums/index.php?showtopic=3287)

Even the Accepted/Rejected situation is documented in this Forum section. A couple of them at http://forum.spamcop.net/forums/index.php?showtopic=2815 and

http://forum.spamcop.net/forums/index.php?showtopic=3059

OK, in all fairness, data probably "hidden" by the "display the last 60 days" setting in this section (most are set to 30 days) .... just went through and killed off ancient redirect pointers of mis-posted stuff ... Guess I'll also go in and change the setting 'here' to go ahead and show "all" posts ...

[turetzsr 0]

...My guess would be that you would be best served by forwarding (as attachment, for e-mails received from the SpamCop Mailhosts system) all relevant information to the deputies at e-mail address deputies <at> spamcop <dot> net.

[Wazoo 0]

However, as already stated, the use of Outlook to send/forward the data may cause further aggravation .... it was due to issues like this that the web-form was created/provided.

[Vanguard 0]

When I saved the attachement (as a .msg file), I looked at it with a text editor and Microsoft is still putzing around with a proprietary format inside this file. Rather than attach the original e-mail (that is getting forwarded as an attachment) as a raw text file containing all the headers and body, I see a bunch of oddball characters printed at the start which supposedly are the headers in some encoded format. There is a ton of encoded crap at the start of the .msg file.

So I used the web submit form. I'd copy the headers from Outlook (View->Options) into the header textbox and the copied the body (which was text already) into the body textbox. It errored with the mismatch on what appears to be the same confirmation code string shown 3 times. So although using the web submit form would eliminate the problem of Outlook using a proprietary formatted attachment (.msg file), it doesn't explain why pasting the headers and body still end up causing an error in the web form.

I am using SpamPal. One complaint that I've made to James Farmer (its author) is that the X-Spampal header has a huge number of trailing tab characters appended onto it. This makes this one line end up wrapping several times and making it look when viewed within a window like there are several blank lines after this header. See http://www.spampalforums.org/phpBB2/viewtopic.php?t=5028. Can such a long header line (which is obvious violating the RFC regarding maximum line length) be screwing up Spamcop's parser?

Is just using Spampal which will insert its headers into the received e-mail cause the problem? Does the confirmation code actually incorporate a hash value similar to when encrypting an e-mail which won't match after SpamPal inserts its headers? Although SpamPal adds headers, it doesn't touch the body so encrypted or digitally signed messages still come through okay. Maybe SpamCop is including the headers in its hash value.

[Wazoo 0]

Go ahead and do the bad thing if I'm wrong, but I have to guess that you've not followed and read the data found in the links provided in my last post.

[Vanguard 0]

Go ahead and do the bad thing if I'm wrong, but I have to guess that you've not followed and read the data found in the links provided in my last post.

27337[/snapback]

I got around to reading all the linked posts (and the linked posts in them, and so on). With so many posts referenced, and with interruptions, it took awhile to get through them all. The synopsis is that the algorithm used to compare the confirmation code in the headers with the one in the body doesn't know how to do string compares (i.e., the string compare is too literal or absolute). The behavior is as though the programmer did a string compare but neglected to include a trim() to eliminate leading and/or trailing whitespace.

The X-SpamCop-Conf header had trailing whitespace added to it. Could be your mail server added it. Could be my mail server added it. The addition of trailing whitespace is not an abnormality. It has nothing to do with which e-mail clients the user is using to respond to the confirmation e-mails from SpamCop. No matter which e-mail client you use, copying the headers means including the trailing whitespace, too. Obviously forwarding an attached copy of the e-mail will also include all of the headers as they originally were, and that includes the whitespace. I'm not running my ISP's mail server so there is no way to check if the X-SpamCop-Conf header came in with all that trailing whitespace. Presumably SpamCop would not be adding it and then have to wonder why it got there, so it is likely that my ISP's mail server is adding the trailing whitespace.

After reading all the linked posts, the most probable successful workaround mentioned was to delete trailing whitespace. So I first deleted the trailing whitespace on the X-SpamCop-Conf header (figuring if that didn't work then I'd have to edit every header to eliminate trailing whitespace from them all). Fortunately, deleting the trailing whitespace from just the X-SpamCop-Conf header got the confirmation process to complete with success.

Apparently this has been a problem since sometime in December if not earlier. Odd that adding a trim() to the string compare should take so long. Oh well, got it working but if trailing whitespace is such a common source of the problem then it should be mentioned in the instructions provided in the confirmation e-mail. I read the pinned posts but I must've missed where it mentions that trailing spaces after the non-blank value in the X-SpamCop-Conf header must be edited out before submitting the confirmation data.

Somehow I doubt my ISP is the only one that might add trailing space onto headers (assuming it wasn't SpamCop's fault). Has there yet been a large adoption of qualifying their mailhosts by SpamCop users? I only came across it because I was editing my preferences and decided to look at the help and click on the tab menues to read more up on SpamCop.

[Wazoo 0]

Critical words are "got it working" ... Congrats!

Julian wrote the code, Ellen is the (at least the most visible) interpretor between user space and database on the MailHost configuration. Once upon a time, she said she'd put together a FAQ when she found the time. Until that time, I and other users can only try to help if possible or point to her guidance to contact her/Deputies if not.

[StevenUnderwood 0]

...such a common source of the problem...

27338[/snapback]

Define "common"...The problem only happens on a relatively few systems that add the whitespace. The whitespace problem does not appear in the bulk of mailhost configurations. Perhaps hte problem should be mentioned is the error response, but it is probably very low on the priority list.

[turetzsr 0]

Define "common"

27341[/snapback]

...Proposed definition: common: Occurs frequently enough to be noticed by participants as more than a rare occurrence.

[Vanguard 0]

Please don't take single words out of context. I said, "if trailing whitespace is such a common source of the problem." I meant "common" in that, for the problem of using the web form (and perhaps, too, for the e-mail method) and it fails, it appears a common (or predominate) solution is to trim the trailing spaces.

I didn't use "common" to say that the problem occurs a lot. I did not say it was a common problem. I meant common in that a solution was the predominate resolution for the problem. From a search (which doesn't guarantee that I see all relevant posts), I saw whitespace and multiple MX records as the likely candidates but I couldn't tell which was the more predominate cause of failure. From your experience, what is/are the most "common cause(s)" for the failure, or what is the predominate cause when there is a failure, or when it fails then what usually makes it fail? I'm not sure how many other ways to say it. The "common" was conditional on the failure occurring regardless of the failure count, low or high. "Common" applies against the population against which it refers, and that population was of failures, not of success and failures.

Trailing spaces, multiple MX records, or quiescent mail servers do not really explain getting BOTH a success and failure when using the e-mail submission method. I realize when you ask someone if they like you that you might get a "Yes and No" answer but computers aren't simultaneously dichotomous in their decisions (even fuzzy logic returns only a single threshold). It was a failure, but it was a success. Argh! Come on, I only have so much hair left.

Thanks for all your help. With it and by reading other (non-pinned) posts, I got it to work. Woohoo!

[StevenUnderwood 0]

Please don't take single words out of context.  I said, "if trailing whitespace is such a common source of the problem."  I meant "common" in that, for the problem of using the web form (and perhaps, too, for the e-mail method) and it fails, it appears a common (or predominate) solution is to trim the trailing spaces. 

27356[/snapback]

OK, to be more precise...When the error message states:

Confirmation codes do not match:

a common source of the problem is that one of the mail servers involved introduced whitespace to one or more of the confirmation codes shown. The common way to search for a solution is to put the error message in the serch engine, in this case "Confirmation codes" searching all forums returns probably 12 cases since April 2004, including the same answers each time.

From your experience, what is/are the most "common cause(s)" for the failure, or what is the predominate cause when there is a failure, or when it fails then what usually makes it fail?

27356[/snapback]

What I have seen when there are problems, it is usually due to getting the complete headers from their mail agent, same as when reporting.

The only problem I know of with multiple MXes is when one of them is dormant but available for disater recovery purposes (like for Postini). In that case the failure is that the secondary MXes did not respond, which makes sense because they are not connected to the internet at the time. But since email will not normally travel that path, it is not such a big deal until they enable the servers. I personally have contacted Postini about this and encouraged them to at least contact the deputies so their Mailhost configuration could be updated to include those IP addresses, but since they have not been updated (as of the last time I checked my settings), it appears that request was not fulfilled. Since I am the admin contact for Postini, I am (supposedly) notified any time that Postini enables those servers so I would know to double check any submissions at that time (and perhaps get the servers added to the config).

[Vanguard 0]

I add the mailhosts today. Since it possible and likely over time that my ISP might add or delete mail servers or change their names, should I periodically rerun the mailhosts check to capture them again? Is there any harm in rerunning the mailhosts check?

[Wazoo 0]

Therein lies one of the issues involvd with Quick-Reporting .... when things change/go wrong, bad reports go out. There is no "MalHost check" ... there is only the "MailHost configuration process" ... so yes, if things change, you'll (possibly) need to add in those new servers, but it's the analysis of the parsing/reporting engine output results that will show if/when it's needed.

[Wazoo 0]

I have no idea if this relates to anything at all, (not knowing what Julian has running) other than the interesting coincidence of the data involved ... "adding trailing spaces" ...

http://forums.invisionpower.com/index.php?showtopic=153352 (probably not accessible, just a future reference point)

Q: MySQL 4.1 has been released, it is a big improvement not just bug fix.

Severals things have changed ie : Fulltext

Does IPB has been tested with mySQL 4.1, what should we take care about before and while converting to 4.1 ?

One answer includes: There are really only 3 "big" changes:

Incompatible change: Binary values such as 0xFFDF now are assumed to be strings instead of numbers. This fixes some problems with character sets where it's convenient to input a string as a binary value. With this change, you should use CAST() if you want to compare binary values numerically as integers

Incompatible change: String comparison now works according to SQL standard: Instead of stripping end spaces before comparison, we now extend the shorter string with spaces.

Incompatible change: TIMESTAMP is now returned as a string in 'YYYY-MM-DD HH:MM:SS' format

The rest are all either special cases (weirdly named tables or special character sets), and don't affect 99.99% of the users out there. And given those changes above, I don't see any problems relating to IPB offhand.

[Ellen 0]

I add the mailhosts today.  Since it possible and likely over time that my ISP might add or delete mail servers or change their names, should I periodically rerun the mailhosts check to capture them again?  Is there any harm in rerunning the mailhosts check?

27392[/snapback]

Trying to answer all the questions that came up in the thread ...

1) the whitespace problem -- yes this is a problem that some users have. I guess I average maybe 2 a month of this problem or similar. Sometimes the whitespace problem is also combined with bad wrapping. Probably bad wrapping is more common.

I have mentioned this to the programmers and so it is known but very honestly I am not sure that it is very high on the priority list.

2) Re a faq -- I did edit a while back one or more of the pinned posts and read thru all of them. I think they say most if not all of what needs to be said -- and in most cases more coherently than I could say it. I will try to find time to read back thru them and see if there is anything I can add.

3) the ISP adding servers/deleting servers -- I would need to see your mailhosts specifically but in general if the new servers have DNS/rDNS/DNS that is rational it should be OK as long as the proper entries are in the domain. However it always helps to keep an eye on the parses and to write to us immediately at the address in the sig line if you see anything screwy. Most of the 'reporting myself" problems we see are from users without mailhosts actually. I cannot remember precisely but I believe there have only been one or two cases where ISP mailserver changes caused bad reporting for users with mailhosts.

I am not tracking on all cylinders today so let me know if I missed something.

[turetzsr 0]

<snip>

I cannot remember precisely but I believe there have only been one or two cases where ISP mailserver changes caused  bad reporting for users with mailhosts.

<snip>

27711[/snapback]

...Yours truly was one of those one or two cases! Seems that my employer's network people will occasionally (although rarely) add what I believe would be called a "routing server" which is (IIUC) neither in any mailhosts configuration nor in the list of trusted servers. Ellen had to rescue me a couple or three times prior to the availability of the MailHosts configuration process and I've had to delete and re-do the MailHosts configuration process.

[Keithj 0]

I can't find this in the "pinned" stuff, so please bear with me.

I've just gone through and "configured" each of my accounts. For every "account configuration e-mail" that I've returned, I seem to have received one message to my "spamcop" address telling me

Hello SpamCop user,

Sorry, but SpamCop has encountered errors:

Headers not found.

It appears you did not provide all email headers. Please consult this FAQ

for more information on getting full headers from your software, then try

again.

Those messages didn't give any detail of which account it was, but by leaving a pause between registrations, it seems to be every one.

And then, almost immediately, another message arrives to the specific address telling me

Thank you for registering your mailhost (keithjhost) with spamcop.

It seems that the error messages were a bit premature, and all is actually well. The accounts appear in my Spamcop "Mailhosts" list.

Have I been doing something wrong, or is this a "feature", and should I worry about it?

[Farelf 0]

Yes, that's confusing and you have cause to be concerned but it does sound like it might, in the end, be okay. Just pay attention to the parser results as you report for a while (use "full" reporting with tech details enabled) and see how it goes. I can't recall anyone reporting quite that sequence of messages before but it may be OK.

[Wazoo 0]

I can't recall anyone reporting quite that sequence of messages before but it may be OK.

Actually, I do recall this "received both a failure and a success message" issue .... At the moment, I will swear that there is an entry with something like those words as either the Subject: line or the "question" ....

[Farelf 0]

Indeed. Merged. Next step to PM O/P. Done. There are other instances that can be found by searching also but went with this - it was the top of the list and it appears to have been resolved for the OO/P.

[Keithj 0]

Yes, it seems to be OK. Reporting is working fine. Case closed, I think.

I do have a new, different problem which I'm sure is unrelated so I'll post that separately.

[Farelf 0]

Yes, it seems to be OK. Reporting is working fine. Case closed, I think. ...

Thanks for the confirmation Keith.