Jump to content
Sign in to follow this  
Wazoo

Graphic & Link added

Recommended Posts

Just a thought, What is the possibility that the small drops on the graph simply represent a period of inactivity in the system, no spam being submitted to the parser; rather than a problem with the system not functioning.  Then again it is hard to know exactly how to read it when you consider the following quote:

35138[/snapback]

I can tell you that the answer to this is no. Going any further than that would be cause to reference another Topic somewhere recently posted, linking back to some recent newsgroup traffic that included links to earlier newsgroup traffic that also included links back into this Forum .. the general gist being that I have no idea what I'm talking about, have no contact with anyone officially connected with SpamCop.net, only post my evil thoughts on anything in publc, and never provide refereneces to back anything up ... I just take that as a sign that every now and them, someone actually reads something that I took the time to type up and post <g>

And forgot to mention, there was no "fix" earlier .... yesterday turned out to be "appliance moving day" ... one oven installed, the replaced oven transported and installed at another home, that oven then to the junkyard ... then the same flow of events on a washing machine, three homes involved in that transfer ... came home, did some catching up, and based on timelines between posts, I slept really well for quite a while <g>

Share this post


Link to post
Share on other sites
I think that possibility is quite low - I don't think I've ever seen the quantity between 0.2 and 2.0 messages per minute average over a 5 minute interval (between 1 and 10 messages total over that 5 minute interval).

35141[/snapback]

I see you point, which is even made stronger when one corrects the error in your logic.

The graph represent reports per second, not minute. The drop in number of reports would be 60 times higher than that which you indicated, further strengthening your argument.

Share this post


Link to post
Share on other sites

Thanks for pointing out my error. Sorry about that. My revision: I think that possibility is quite low - I don't think I've ever seen the quantity between 0.1 and 2.0 messages per second average over a 5 minute interval (between 30 and 600 messages total over that 5 minute interval). As of 11:06:01 EDT -0400, total spam (processed by that server) over the past 24 hours was 337,016 messages, or 14,042.3 messages per average hour, or 234.039 messages per average minute, or 3.9006 messages per average second. Based on the number of SpamCop Reports sent over a similar period (from Report ID 1541889229 submitted at 12:25:19 -0400 yesterday to Report ID 1542628992 submitted at 11:17:54 -0400 today), 739763 Reports over 22 hours 52 minutes 35 seconds (82,355 seconds), that calculates to 8.9826 Reports per average second, or 2.3028 Reports per average spam message processed. Given the height of the blue line relative to the height of the green bar, I'd estimate that in the past day, "only a small fraction" was actually the whole, and that EXACTLY ZERO spam messages and Reports were processed during the "drops" I identified.

Share this post


Link to post
Share on other sites

The following was the caption from the 24 hour view of the graph:

Average spam: 3.8 messages per second Max spam: 7.0 messages per second

Total spam (last 24 hours): 326841 messages [sat Oct 29 14:36:01 EDT 2005]

This would sure support your conclusions that the statement, "only a small fraction" was actually the whole; and if not the entire total, much closer to the total than to a "small fraction"

Jeff, thank you for your validation using the report numbers as a point of reference.

Share this post


Link to post
Share on other sites

You're welcome. One further observation: if there was any data at all, I would expect the bottom line on the graph to be have some green and/or blue; instead, it is entirely black.

Share this post


Link to post
Share on other sites

... and then there was a dip to around 1.4 between around 16:00 and 16:05 EDT -0400 (between around 20:00 and 20:05 EDT -0400), just as if to prove me wrong. :)

Another drop, about 18:00-18:10 EDT -0400, 22:00-22:10 UTC -0000

30 Oct 2005

... and another drop about from 02:15 to 02:25 EST -0500 (about from 07:15 to 07:25 UTC -0000)

... and another drop about from 09:50 to 10:00 EST -0500 (about from 14:50 to 15:00 UTC -0000)

Share this post


Link to post
Share on other sites
... and another drop about from 09:50 to 10:00 EST -0500 (about from 14:50 to 15:00 UTC -0000).

35198[/snapback]

Boy, the things you notice when you have full tracking :)

Share this post


Link to post
Share on other sites
full tracking

35204[/snapback]

?

Share this post


Link to post
Share on other sites
?

35205[/snapback]

Full Tracking == we are watching the graphs more closely than normal due to the live view being at the top of the forums.

My theory being that there may have often been these quick drop outs but they were never noticed before because the page was "hidden" from out view except when we had a problem and thought to check it.

I have a similar circumstance with a piece of equipment at work we used to only look at when there was a problem (rarely). We now check these logs daily. This one piece of equipment has a problem viewing the logs about twice a month. We are working with the manufacturer and it appears to be a bug in their code (The log does not fit inside the SNMP data stream).

Share this post


Link to post
Share on other sites

What you describe as 'quick drop outs' have never been this frequent .... thiese things you are seeing now with such frequency didn't start until after last "major" hardware upgrade .... the 'massive' failure that followed really messed things up. It is just my opinion, but I still believe that the "golden" code was lost that day.

There was once a system, 8 of which existed in the world. One genius programmer. He arrived at the site I supported with the latest "software upgrade" in hand. Spent Monday loading in some updates via mag tape, then typing in yet more code. Talking to analysts that used the system, he began tossing some new code to take in their suggestions. Monday evening, he dumped that hard drive platter to a new mag tape. Tuesday morning, he grabbed Backup drive platter set (these were the 90+ pound stacks of plated 20"+ disk packs placed into the washing machine sized hard drives) . loaded the tape from Monday evening to bring that disk pack up that stage ... then spent the rest of the day making more code changes. Tuesday evening, dumped Backup platter #1 to mag tape ... Wefnesday he grabbed Backup platter #2, loaded the tape from Tueday evening and spent the day making more code changes. repeat and rinse through Friday afternoon. ... Had the analysts set down, run the system through it's changes, checking out all the 'new (undocumented)' changes based on their input .. everybody happy ... he packed his stuff grabbed the "last" mag tape with "all the changes made" and basically hit the road to hit the next site.

Come the next Monday, call comes to my maintenance shop ... this system is not usable .... Posted elsewhere in this Forum is a tale of the day I learned Fortran ... this incident is what caused that to happen. Only after a lot of research, trial and error, conversations with the analysts, a few phone calls did the impact of the above described sequence become apparent. First problem I had to resolve ... which disk platter set/mag tape actually held what version ... not done was bringing all that stuff up to the "same" stage, so all disk packs involved had different versions of some sotware, not all of them had "all" the files found on the "last" version ..... only known fact was that "all" diskpacks had been tried, none of them "worked" .. confusing as everyone was happy the previous Friday <g>

While waiting for the mad genius programmer to get somewhere that he could be contacted, news came in that he had died of a heart attack in Athens ....

Back to "computers are electronic, aren't they?" .... my orders were the "fix it, call me when you're fone" .... this led to the "great security incident" that I believe I also posted elsewhere .... although working in the situation where I was read onto a program that didn't exist, so the clearance didn't exist, and paperwork says that I can't ever admit to knowing about it ... when it turned out the the issue boiled down to that each site had it's own password .. that passwird was hard-coded into the siftware ... and when I came into the vault on a Marine guarded secure floor in a Marine guarded secure building on a Marine guarded secure installation <g>) with a print-out of the section of code with the issue identified and the simple question of "what's 'our' password?" ..... turns out I wasn't cleared to 'that' level at the time <g> .. never mind the question of how I'd come across that data to begin with .... trying to explain to the security folks that you had to have the computer knowing what the password was if it was going to check if you were using the right one or not was silliest (yet scariest) discussions in my life.

The funny ending of that story was that the genius programmed had finished updating most of the sites, the remaining two were set copies of (something) from the development office .... but within the month, it boiled down to that the site I was supporting was the only one still functional. The LCDR running the show there would be only too happy to help out all the other sites and the Development office if they would only "officially" ask for his help in fixing "their" system <g> Aaahh, the politics and games involved .... he got promoted, I got yet another medal to have to deal with on a dress uniform, so I guess it worked (especially when compared to those 20 to 50 years in prison discussions <g>)

Share this post


Link to post
Share on other sites

... and two more drops about from 00:55 to 01:05 EST -0500 (about from 05:55 to 06:05 UTC -0000) and about from 05:05 to 05:15 EST -0500 (about from 10:05 to 10:15 UTC -0000).

... major dip and then drop about since 11:20 EST -0500 (about since 16:20 UTC -0000). http://mailsc.spamcop.net/ showed the following:

An error occurred while processing your request.

Reference #97.19910c40.1130862980.1a4f47a

It appears to be better now about since 11:40 EST -0500 (about since 16:40 UTC -0000)

Share this post


Link to post
Share on other sites

... and then there was another drop about from 00:50 to 01:05 EST -0500 (about from 05:50 to 06:05 UTC -0000)

(turetzer) ...And another drop now (approx 9:00 EST, 1400 UTC/GMT)

My estimate is about since 08:50 EST -0500 (about since 13:50 UTC -0000)

(turetzer) ...It seems to be back in business, now. <g>

Yes, it appears to be back (but very busy catching up) about since 09:20 EST -0500 (about since 14:20 UTC -0000)

... and then there was another drop about from 11:20 to 11:30 EST -0500 (about from 16:20 to 16:30 UTC -0000)

... and then there was another drop about from 18:35 to 18:45 EST -0500 (about from 23:35 to 23:45 UTC -0000)

Share this post


Link to post
Share on other sites

... and so it finally came to pass, after days and days of careful monitoring, that the graph at http://forum.spamcop.net/forums/index.php?showtopic=5247 finally came to be a continuous graph, meaning a whole day of continuous uptime for the SpamCop Parsing and Reporting Service. And there was much rejoicing!

Thanks to everyone who helped to make this possible!

Edited by Jeff G.

Share this post


Link to post
Share on other sites

... and then the rejoicing died down, as there was another drop about from 11:40 to 11:45 EST -0500 (about from 16:40 to 16:45 UTC -0000).

Share this post


Link to post
Share on other sites

... and then there were two more drops about from 00:25 to 00:40 EST -0500 (about from 05:25 to 05:40 UTC -0000) and about from 01:10 to 01:20 EST -0500 (about from 06:10 to 06:20 UTC -0000)

... and then there was another drop about from 12:35 to 12:45 EST -0500 (about from 17:35 to 17:45 UTC -0000)

The image is again working, along with the HTTP Daemon on the underlying server alpha.cesmail.net.

Share this post


Link to post
Share on other sites

Downtime is downtime. SpamCop has many thousands of users, don't they deserve to know the truth?

At present, apha.cesmail.net is responding to ping, but not HTTP. That's why the graph isn't showing up. I've sent notifications to JT.

Share this post


Link to post
Share on other sites

I'm sorry, in my haste to notify you I misspelled "alpha" as "apha".

Share this post


Link to post
Share on other sites

... and then there was a long dip-drop-dip period about from 04:30 to 06:20 EST -0500 (about from 09:30 to 11:20 UTC -0000), totaling about 1 hour 50 minutes, followed by about 35 minutes of recovery (processing of the backlog).

... and then there was another drop about from 16:25 to 16:40 EST -0500 (about from 21:25 to 21:40 UTC -0000)

Share this post


Link to post
Share on other sites
Downtime is downtime.  SpamCop has many thousands of users, don't they deserve to know the truth?

<snip>

35524[/snapback]

...Sure ... that's why the graphic is there and (for those of us who can't see the details because it is so small) the link to the larger graphic. IOW, I'm inclined to agree with StevenUnderwood's implied suggestion (that we no longer benefit from additional detail) on this.

Share this post


Link to post
Share on other sites

Another nice advantage of the graphic is that at this present time it is indicating a major system down period. Clicking on the graphic does not work (since the system is down) but the graphic still somehow does manage to display the down situation.

So may thanks again goes to Wazoo for adding it in.

No need to wonder at why I can't report spam right now, the little graphic says it all.

Share this post


Link to post
Share on other sites

As pointed out, the actual graphic is the image displayed via the cesmail system (JT's hardware) but the link is pointed to the spamcop.net system such that (when it works) the display is still 'wrapped' by the www.spamcop.net web page screen, with all the links found there.

Share this post


Link to post
Share on other sites

I like the "wrapper" here better than there for the following reasons:

  • It's not there, so it's not affected by there's outages.
  • In Firefox 1.0.7, it maintans its horizontal and vertical position on refresh here, but not there.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×