SpamCop Discussion > [Resolved] Reporting System Recovery

Full Version: [Resolved] Reporting System Recovery - 27 Mar 2009

SpamCop Discussion > Discussions & Observations > Announcements

Wazoo

Mar 27 2009, 02:24 PM

Forum and newsgroup traffic galore .... so playing the catch-up mode here ....

Bottom-line, many issues after the upgrade/maintenance effort yesterday. Problem areas seem to include a crashed hard-drive, the load-balancing and connections involved in getting the various databases and handshaking working betwwen all the various components of the Parsing & Reporting System.

Impacts have been seen to include Parsing & Reporting actions, MailHost Configuration issues, Parsing results ... strangely, login issues have not been complained about. (OK, I'll take that back, the VER connection has issues. Once again, this is a backdoor hack that connects between the Cisco/IronPort systems on the west coast to JT's E-mail system on the east coast, amd based on provided dialog, the 'problem area' is on the Parsing & Reporting side of the house.)

Current situation at the time of this posts seems to be that the majority of the handshaking has caught up, most of the backlog has been waded through, but ... there may still be some bugs floating around with the new software now in place. Cisco/IronPort engineers allegedly working to resolve these issues.

A few of the posts made here by Don:

http://forum.spamcop.net/forums/index.php?...ost&p=70118

QUOTE

Yes, something has gone wrong in the Mailhosts system.

SpamCop suddently doesn't recognize its own servers.

Not one single thing I can do about it, either. I mean on my own. Engineers have to fix this.

I have feelers out to find out the best way to approach it.

http://forum.spamcop.net/forums/index.php?...ost&p=70108

QUOTE

It's beginning to look like there are bugs that need to be worked out. Hard to say for sure.

We're on it and so are the engineers.

We don't know anything yet, but testing proceeds apace.

http://forum.spamcop.net/forums/index.php?...ost&p=70134

QUOTE

It looks like manual reporting is back to normal now.

There are still a couple of small issues to be fixed, but all in all, everything is looking good.

Thanks for your patience!

This last post by Don made something like 20 or so minutes prior to this Announcement post.

rconner

Mar 27 2009, 04:04 PM

Just to get it on the forum, I'm (still) having problems seeing my held mail. Cookie login to mailsc works fine, but when i hit Held Mail it says "Cannot find login information. Not logged in?"

Manual login from this page is rejected due to bad pw or user ID, and I can't reset the password (my user ID is not found).

Webmail works OK, I can see my held spam from there.

I've already e-mailed twice on this, but just wanted to get it up on the board.

-- rick

StevenUnderwood

Mar 27 2009, 04:52 PM

QUOTE(rconner @ Mar 27 2009, 05:04 PM)

Just to get it on the forum, I'm (still) having problems seeing my held mail. Cookie login to mailsc works fine, but when i hit Held Mail it says "Cannot find login information. Not logged in?"

I do have a topic in the Reporting forum and a link in the Email forum to that post about this issue.

Glad to see you have the same issue from the cookie login... make the issue universal as I use the popup login and see the same thing.

Lking

Mar 27 2009, 05:36 PM

QUOTE(StevenUnderwood @ Mar 27 2009, 05:52 PM)

Glad to see you have the same issue from the cookie login... make the issue universal as I use the popup login and see the same thing

Had trouble an earlier but my cookie login seems to work now. When I had a problem, it did work on the third try.

Sure am glad I'm on this side of the problems and don't have a bunch of tech savvy geeks watching every move. <g>

Wazoo

Mar 27 2009, 09:22 PM

From the spamcop newsgroup;

QUOTE

From: RW <nobody[at]spamcop.net>
Newsgroups: spamcop
Subject: All Fixed??
Date: Fri, 27 Mar 2009 18:23:59 -0600
Message-ID: <gqjqmu$dav$1[at]news.spamcop.net>
NNTP-Posting-Date: Sat, 28 Mar 2009 00:23:58 +0000 (UTC)
Xref: news.spamcop.net spamcop:171252

Reports from Engineering are they found a problem between the master and
slave databases. This has been fixed now, so the parsing
problems/errors should be gone.

They also found a coding in the held mail coding, which has been fixed.
Held mail '500' errors should now be gone

They've also fixed the apnic problem and abuse.net problem, so those
should be fixed.

Let us know if you're still seeing any error or come across any new ones.

Thanks,

Richard

rconner

Mar 27 2009, 09:37 PM

QUOTE

They also found a coding in the held mail coding, which has been fixed.
Held mail '500' errors should now be gone

Don't know what problems got fixed, but they weren't mine. As noted elswhere, I am still unable (at 22:30 EST) to show my Held Mail in VER. Very annoying. Pasted-in spam parses OK, and webmail continues to work

-- rick

Wazoo

Mar 27 2009, 10:19 PM

QUOTE(rconner @ Mar 27 2009, 09:37 PM)

An unanswered cross-posted into numerous newsgroup post, made in response to a Steven Underwood post;

QUOTE

From: "Michael R N Dolbear"
Newsgroups: spamcop,spamcop.help,spamcop.mail,spamcop.geeks
Subject: Re: Scheduled Maintenance Thursday March 26, 2009 - (SpamCop 4.5.0-084 Release)
Date: Fri, 27 Mar 2009 21:43:31 +0000 (UTC)
Message-ID: <01c9af1e$f71073a0$LocalHost[at]default>
References: <gq9cv6$vsk$1[at]news.spamcop.net> <gqgcs3$b1o$1[at]news.spamcop.net> <gqh34h$3tr$1[at]news.spamcop.net> <gqh60n$g71$1[at]news.spamcop.net> <gqhbmu$23k$1[at]news.spamcop.net>
X-Trace: news.spamcop.net 1238190211 14862 62.64.205.119 (27 Mar 2009 21:43:31 GMT)
NNTP-Posting-Date: Fri, 27 Mar 2009 21:43:31 +0000 (UTC)
Xref: news.spamcop.net spamcop:171251 spamcop.help:94934 spamcop.mail:22531 spamcop.geeks:28420

Steven Underwood <nobody[at]spamcop.net> wrote

> While I see that the reporting site appears to be up, I am getting the
> following error trying to reach the VER site from my logged in account:
>
> Cannot find login information. Not logged in?
>
> VER tends to be a lot quicker getting messages into the queue as opposed to
> forwarding from webmail, but I will revert back to that if necessary.

I'm still getting that at 1800 GMT , but only if I go to (from SC mail or bookmark)

http://mailsc.spamcop.net/reportheld?action=heldlog

(and I can look at my old reports so it does know I'm logged in)

If I login at the "cannot find login ..." message I get sent to a different VER url

http://www.spamcop.net/reportheld?action=heldlog

Which works.

What's going on ?

Mike D

rconner

Mar 27 2009, 10:27 PM

That's pretty much what has been going on with me for the past 24+ hours.

-- rick

dbiel

Mar 27 2009, 11:14 PM

I just tested the VER held mail link and it is working fine for me

StevenUnderwood

Mar 28 2009, 12:07 AM

QUOTE(dbiel @ Mar 28 2009, 12:14 AM)

I just tested the VER held mail link and it is working fine for me

Glad it is working for you... Cookie login does seem to be working but "HTTP basic auth" login is still not responding with the same: Cannot find login information. Not logged in? error.

Wazoo

Mar 28 2009, 02:24 AM

Posts made recently within this Forum, reference the VER Reporting mode;

QUOTE

We're still having problems with reporting from the Held Mail folder when logged into the reporting side.

The engineers are on it and an emergency fix is planned. I don't know if that means daylight hours on Saturday, or if we have to wait until Monday.

Hope springs eternal.

- Don D'Minion - SpamCop Admin -

Wazoo

Mar 28 2009, 06:10 PM

From the spamcop newsgroup;

QUOTE

From: SpamCop Admin
Newsgroups: spamcop
Subject: Re: All Fixed??
Date: Sat, 28 Mar 2009 05:43:17 -0600
Message-ID: <823ss4dkddfmgnp0l1afp41dg6paue6ivp[at]4ax.com>
References: <gqjqmu$dav$1[at]news.spamcop.net> <gqk9q2$h5f$1[at]news.spamcop.net>
NNTP-Posting-Date: Sat, 28 Mar 2009 11:43:17 +0000 (UTC)
X-No-Archive: yes (damn it!! Why does an Admin do this?)
Xref: news.spamcop.net spamcop:171257

Opinicus wrote:
>-It's 04:40 UTC and I'm still getting "Cannot find login information. Not
>-logged in?" when I try and enter from
>-http://mailsc.spamcop.net/reportheld?action=heldlog

I'm seeing the same thing when I'm logged into mailsc and try to get
to Held Mail.

I'll file a bug to get that fixed.

In the meantime, if you're willing to accept cookies, you can log into
http://www.spamcop.net/

You can get to Held Mail from there, or you can get to your Held Mail
folder if you log into your account at
http://webmail.spamcop.net/horde/imp/login.php

- Don D'Minion - SpamCop Admin -

rconner

Mar 28 2009, 11:25 PM

Still having my original problem 48+ hours on. I tried a login at www.spamcop.net as suggested above, and then clicked "Held Mail," then got a very fancy looking "500 Internal Server Error" page. I refreshed a couple of times, and finally got my held mail to show up.

So, limited victory, but I would prefer to have mailsc back online and working.

-- rick

StevenUnderwood

Mar 29 2009, 08:33 AM

QUOTE(rconner @ Mar 29 2009, 12:25 AM)

Confirming... I am seeing the same thing and also just saw the Report spam link pointing to my just reported spam. All the original issues I have seen remain.

1. http://mailsc.spamcop.net/ - Held Email tab results in solid "Cannot find login information. Not logged in?" error.
2. http://www.spamcop.net/ - Held Email tab results in intermittent "500 Internal Server Error" probably for the same reasn (connection issues to mail storage server?).
3. Aftrer reporting a spam message, intermittent result is showing a link to another spam to report, usually in a color indicating I have already been there, to the spam I just reported. I personally have seen this happen about 50% of the time, though I do not report very high volume any longer (thank you grey listing).

rconner

Mar 29 2009, 12:32 PM

OK, at 13:20 EST I cannot pick up held mail at all. Neither HTTP (mailsc) nor cookie login (www.spamcop.net) is working for me. THe HTTP login continues to behave as described in earlier posts, no changese there. The HTTP login doesn't seem to recognize the 12-hour cookie I set yesterday, and won't accept my user ID & password (the HTTP login does accept it, it recognizes me but won't give me my held mail).

So, I'm pretty much out of the spam reporting business right now, except for spam that sneaks past the Held Mail queue.

Thanks, its been grand.

-- rick

DavidT

Mar 29 2009, 02:16 PM

QUOTE(rconner @ Mar 29 2009, 10:32 AM)

So, I'm pretty much out of the spam reporting business right now, except for spam that sneaks past the Held Mail queue.

There are at least two other methods that should enable you to access your Held mail.

1. webmail access to your SpamCop email account
2. IMAP access to your SpamCop email account

Using either of those, you should be able to keep reporting anything you choose to.

DT

Wazoo

Mar 30 2009, 07:00 AM

Yet another post carried up from the Reporting Help Forum section, made about an hour before this post;

QUOTE

QUOTE(StevenUnderwood @ Mar 29 2009, 09:53 AM)

there IS a problem with the reporting system sending you back to the link you just processed.

Yes, there is. I just filed a high-priority bug ticket to get it fixed.

In other news... The "500 Internal Server" error should be handled by a hot-patch later today.

It's 04:00 in California right how, so "later" means many hours from now.

- Don D'Minion - SpamCop Admin -

Lking

Mar 30 2009, 03:13 PM

From SpamCop News

QUOTE

Subject: SpamCop 4.5.0-101 Hot Patch Deployment

Date/Time of Maintenance: Monday, March 30, 2009, 2:00pm PDT

Expected Duration: 1 hours

Impact: Our SpamCop cluster will have intermittent availability during
the 1 hour window.

Customer Impact: Direct customers of our SpamCop service will notice
intermittent connectivity during the downtime.

This hotpatch addresses the mailsc/held mail/etc errors of the last
couple of days.

Please propagate as needed.

Ellen
SpamCop

StevenUnderwood

Mar 30 2009, 05:25 PM

QUOTE(Lking @ Mar 30 2009, 04:13 PM)

From SpamCop News

And this appears to be working for me for the last 30 minutes or so. Have not had any spam to report to this point, however, but I can at least get to the Held Mail tab.

rconner

Mar 30 2009, 09:46 PM

QUOTE(StevenUnderwood @ Mar 30 2009, 06:25 PM)

And this appears to be working for me for the last 30 minutes or so.

I can confirm that all appears to be working for me now (at any rate I can get spam from Held Mail and report it). Thanks to all involved for the effort. Now, if I could just get my ISP to keep the damn DSL nailed up...

-- rick

This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.