Jump to content
Sign in to follow this  
docsmooth

Still listed after 48 hours, only "spamtrap"

Recommended Posts

WB / John - I strip all <scri_pt> tags from incoming email, and block emails with offsite image links (image linking can be whitelisted). This is done before the user can see the email, no matter how they access it. I've seen those tags in many places before, which is why I block those pieces outright. Outlook 2000 will render them just as well as any browser.

Someone mentioned the 48 hour thing in my original post. I just want to clarify that statement: I waited 48 hours after being listed AND checking my outbound mail traffic for spam (tcpdump of past email, not queues).

I'm guessing, based on (I think) Ellen's post, that I didn't go far enough back in my traffic - as far back as I went, there was no spam, because as far back as I went (1-2 days at the time), I must have already had been blacklisted. Because I didn't look far enough back in time, I didn't see any spam, and therefore originally assumed I was being BLed for NDRs (I've had a few complaints for that, and for our order confirmations).

i'm still working on closing my NDR hole, but can't find much information regarding that on IIS6 yet.

Not Exchange, Just IIS6 SMTP.

Again, thanks all for your help - I'm continuing to monitor this thread (obviously) for anything else you say I don't want to miss. :)

Share this post


Link to post
Share on other sites
If that is the case, would it not make sense to change the reporting window to two days instead of 3

So would it be safe to say that the following quote is totally incorrect?

16302[/snapback]

Well there have been discussions about 2 vs 3 days for reporting but right now it is still at 3.

Ugh where is the rest of what you asked? .... oh no the quote from the faq is correct. Once spam is reported it is counted against the IP for a week -- i.e. it ages down over the period of a week. That is different from how old the spam can be before the user reports it. I think it makes sense -- maybe it doesn't.

Share this post


Link to post
Share on other sites
Well there have been discussions about 2 vs 3 days for reporting but right now it is still at 3. 

Ugh where is the rest of what you asked? .... oh no the quote from the faq is correct. Once spam is reported it is counted against the IP for a week -- i.e. it ages down over the period of a week. That is different from how old the spam can be before the user reports it.  I think it makes sense -- maybe it doesn't.

16399[/snapback]

So is that to say that there is or there is not a specific 48hour clock that keeps getting reset each time a report is filed so that when 48hours has past the IP address is automaticly delisted; Or is it just a generalization and the complex calcualtion has sole control over when a site is delisted (unless manually changed by management intervention).

Share this post


Link to post
Share on other sites
So is that to say that there is or there is not a specific 48hour clock that keeps getting reset each time a report is filed so that when 48hours has past the IP address is automaticly delisted; Or is it just a generalization and the complex calcualtion has sole control over when a site is delisted (unless manually changed by management intervention).

As stated somewhere in this, there is a "weighting" factor applied in that complicated bit of mathematical formula. This "weighting" factor is directed towrds the decision threshold of where between the one-half hour and the 48 hours maximum come into play. That ISP that just yesterday signed up a spammer, spammer started a spew, and this IP nailed the account quickly .... this IP would tend to go to the one-half hour range. That ISP that allows the spew to continue, thus having an IP that hits the list, spammer notices and moves the spew elsewhere until that IP is unlisted again and moves back, spewing until this IP is isted again .... now this IP is going to tend to go the 48 hour maximum range .... not sure whether to liken it to inertia or a bit of hysteresis.

Share this post


Link to post
Share on other sites
<snip> the quote from the faq is correct. Once spam is reported it is counted against the IP for a week -- i.e. it ages down over the period of a week. <snip>

16399[/snapback]

My question still remains (I know that I am picky) based on Ellen's above statement, can it still be said that there is a true 48hour max clock, or can spam reports that are still being counted for up to 1week cause that the delisting period to extend beyond the 48hour period. I do believe that the 48hour rule is a good general rule of thumb, but I still contend that we are incorrectly stating that all IP's will be delisted within 48hours of the last report.

Share this post


Link to post
Share on other sites

OK, how about bringing the 2% threshold back into the picture .. call that the indicator that spew has stopped, with the weighting factor applied to the count-down clock from that point?

Share this post


Link to post
Share on other sites
So is that to say that there is or there is not a specific 48hour clock that keeps getting reset each time a report is filed so that when 48hours has past the IP address is automaticly delisted; Or is it just a generalization and the complex calcualtion has sole control over when a site is delisted (unless manually changed by management intervention).

16401[/snapback]

If there are 48 hours with no reports then the IP gets delisted. If reports resume then whether there is or isn't a listing/relisting is a function of the formula in the faq which evaluates the number of active reports and their weighting vs traffic.

Share this post


Link to post
Share on other sites
My question still remains (I know that I am picky) based on Ellen's above statement, can it still be said that there is a true 48hour max clock, or can spam reports that are still being counted for up to 1week cause that the delisting period to extend beyond the 48hour period. I do believe that the 48hour rule is a good general rule of thumb, but I still contend that we are incorrectly stating that all IP's will be delisted within 48hours of the last report.

16409[/snapback]

The 48 hour rule determines the delisting. No reports for 48 hours, then the IP delists.

However *if* reports start back up then the new reports with their weighting plus any other reports which are age rated are then counted towards relisting.

Share this post


Link to post
Share on other sites
If there are 48 hours with no reports then the IP gets delisted. If reports resume then whether there is or isn't a  listing/relisting is a function of the formula in the faq which evaluates the number of active reports and their weighting vs traffic.

16419[/snapback]

Thanks for the reply.

Please correct me if this statement is wrong (I know that I am beating a dead horse, but it is important)

ISP finds problem on server and corrects it. Result is no more spam is being sent.

Just prior to the fix a million spam messages were sent.

The IP is delisted 48hour after the fix

SpamCop customers report 200 spam messages 2 - 16 hours after the delisting (within the 3 day reporting window) Note: messages were sent 52 hours before the delisting)

Based on the previous statements it should be safe to say that the IP would NOT be relisted because of the 200 newly filed reports but it would most likely be relisted if a single newly sent spam message was reported or spamtrap hit after the initial delisting.

As long as the above statments can be considered valid, then this thread is now terminated as all questions have been answered. If it is not valid then the question remains unanswered.

My sumarization of what has been posted:

1) There is a complex formula that determines if and when and for how long an IP address is listed.

2) All reports are considered sent on the date and time that the spam message was originally received, NOT when it was reported.

3) There is a separate 48hour delisting clock that will automaticly delist an IP if there have been no reports of spam messages sent and received within that last 48hour window. New reports filed during this time window will not extend the listing as long as the messages themselves were received prior to the last 48hour window. The actual time that the reports are filed is meaningless (as long as they are filed in the 3 day filing window).

4) If the complex formula says that a IP should be listed, but the 48hour clock says it is OK to delist, then the IP will be delisted and remain unlisted until such time as the 48hour clock rule has been violated at which time the complex rule comes back into play.

Share this post


Link to post
Share on other sites

Ellen said nothing to back me up, but I still say you are missing the part of the formula that involves setting the range from a minimum of one-half hour to the maximum of 48 hours. There's the threshold, there's the weighting that include time, quantity, and the good/bad ratio, and of couse, time.

You example of reporters kicking in the additional 200 reports doesn't seem to take into account that (in theory) the "measured" e-mail traffic would also have been incrementing a bit, thus adjusting the threshold and weighting just a bit, on top of the age of those reports causing a bit of adjustment.

Share this post


Link to post
Share on other sites
Ellen said nothing to back me up, but I still say you are missing the part of the formula that involves setting the range from a minimum of one-half hour to the maximum of 48 hours.  There's the threshold, there's the weighting that include time, quantity, and the good/bad ratio, and of couse, time.

You example of reporters kicking in the additional 200 reports doesn't seem to take into account that (in theory) the "measured" e-mail traffic would also have been incrementing a bit, thus adjusting the threshold and weighting just a bit, on top of the age of those reports causing a bit of adjustment.

16429[/snapback]

It seems that I have a difficult time expalining myself.

I have been focusing on a single point, that being the 48hour and you are delisted rule.

I have no problem accepting that 48hour may be a good generalization based on all the varrious factors involved.

I am not missing the point about the part of the formula that involves setting the range from a minimum of one-half hour to the maximum of 48 hours, it just is not important as related to my question.

My question deals with the range starting with the maximum of 48hours going up. and trying to validate if it should be considered (in which case the maximum of 48hours is false) or if it should be ignore because a max of 48 hours means just that - 48hours.

My example of 200 messages was intentionally structured to be out of the norm as I have yet to see any posted reports coming even close to that number.

The example is theoritical and the specific number is meaningless.

To restate the question differently.

If 10,000 users report spam from one IP address more than 48hours after it was originally sent and it is also true that the IP in question has managed to stop all spam from going out so that NO spam has been sent for 48hours will that IP be listed or not? My contention is that complex formula can override the 48hour rule resulting in the IP being listed for more than 48hours after the last spam left the IP and/or arrived at the original destination.

Edited by dbiel

Share this post


Link to post
Share on other sites
It seems that I have a difficult time expalining myself.

I have been focusing on a single point, that being the 48hour and you are delisted rule.

I have no problem accepting that 48hour may be a good generalization based on all the varrious factors involved.

I am not missing the point about the part of the formula that involves setting the range from a minimum of one-half hour to the maximum of 48 hours, it just is not important as related to my question.

My question deals with the range starting with the maximum of 48hours going up. and trying to validate if it should be considered (in which case the maximum of 48hours is false) or if it should be ignore because a max of 48 hours means just that - 48hours.

My example of 200 messages was intentionally structured to be out of the norm as I have yet to see any posted reports coming even close to that number.

The example is theoritical and the specific number is meaningless.

To restate the question differently.

If 10,000 users report spam from one IP address more than 48hours after it was originally sent and it is also true that the IP in question has managed to stop all spam from going out so that NO spam has been sent for 48hours will that IP be listed or not?  My contention is that complex formula can override the 48hour rule resulting in the IP being listed for more than 48hours after the last spam left the IP and/or arrived at the original destination.

16433[/snapback]

It shouldn't be.

Share this post


Link to post
Share on other sites
It shouldn't be.

16437[/snapback]

Unfortunately that is not the same as it will not be listed which brings us back to the statement that "An IP will automaticly be delisted when it stops sending spam, this normally will take place in 48 hours or less.

"It shouldn't be" does no support the use of a max of 48 hours in our statements.

Just a thought in case my reply is in left field due to my broken crystal ball,

It seems that I have a difficult time expalining myself. <snip>
I made the assumption that your reply was not in responce to the above portion of by post but was rather in reply to the following portion
If 10,000 users report spam from one IP address more than 48hours after it was originally sent and it is also true that the IP in question has managed to stop all spam from going out so that NO spam has been sent for 48hours will that IP be listed or not?
It shouldn't be.

16437[/snapback]

Thank you for your patience and replies. Edited by dbiel

Share this post


Link to post
Share on other sites
To restate the question differently.

If 10,000 users report spam from one IP address more than 48hours after it was originally sent and it is also true that the IP in question has managed to stop all spam from going out so that NO spam has been sent for 48hours will that IP be listed or not?  My contention is that complex formula can override the 48hour rule resulting in the IP being listed for more than 48hours after the last spam left the IP and/or arrived at the original destination.

I'm going to stick with the "threshold" as being part of the scenario. Let me try to add some of the other factors in. Out of the thousands of computers I've had my hands on in just my local area over the last few years, and even though at least 95% of those computer owners know what spam is and recieve some daily, I am the only spam reporter (and noting actually that I have hardly used SpamCop myself in quite a while) ....

So to try some numbers, 2,000 known computer users that all receive spam .... one person that submits one of those same spams to/through the SpamCop reporting tool. One complaint. The SpamCop analysis /monitoring tools only happen to see 30% of that total traffic from the origination IP that the one report identified. So we now have one report out of 600 items of e-mail traffic.

Now, we move these same number down to the next little town, where we find that there are two people that actually report via SpamCop, but only one of these people gets the same spam as I did. So now we're up to 2 reports against this same IP, but we're not close to the 2% treshold of spam to traffic seen.

Now, just maybe, the next town is a bit bigger, and there are 50 people there that use SpamCop. We might be getting close to the threshold of listing this IP. But, while doing all this checking and calculation, the typical spammer scenario has had more spam / seen e-mail traffic leaving this IP, so the numbers have been changing.

So I'm stating here that for 10,000 reports to be made against that one IP, and let's go worst case, all reported on Monday morning from the single spam spew run completed during the late hours Friday night/early hours Saturday morning ... I would suggest that it would be "possible" for that IP to get listed sometime Monday morning, but "it depends" .... The weighting of the spam reports due to age are also leveraged against the continuing "seen e-mail traffic" from that IP that is not getting reported as spam. Somewhere, those two lines will intersect, and the IP gets unlisted. If there is much traffic, it'll be a short listing. If little traffic (say the compromised server for some small company that was fixed Saturday morning when the system bog was noticed) then it's more likely that the full 48 hours will be in play.

If this IP is not listed because of the results above, then the conditions are set that the next "fresh" spam will be the one that changes the condition and does get it listed. So, now the catch is ... is there a reporter out there that although reporting the same spam, this user's ISP has a system clock set wrong .... do we have a reporter that noted the date was a bit off and "fixed" it so it would be reported .... do we have a report made with some massively screwed up header lines due to the way the user cut/pasted/forwarded the data .... do we have a condition made famous in the [at]Home days where the e-mail had been sitting on an ISP's server, but that server had just been put back in the rack yesterday and was now merrily delivering that year-old e-mail (doesn't fit the above scenario, just an extreme example)

The spam reported via SpamCop is just a small sampling of the actual spew being sent, but the measured "seen e-mail traffic" is also a small sampling, as this data is only collected from some servers around the world, thus the vagueness on developing some hard numbers. I one was to go with your 10,000 reports and reflect that this reporting acion is done by so few of the recipients, might we also be back to the 1,000,000 spam actually being sent? And if this is anywhere being true, might not we be back to "is 48 hours enough punishment?" <g>

Here's another bit of a question, perhaps also relating to what you're trying to dig up ... when trying to analyze the "status" of a particular IP, running the numbers through this formula will provide a snapshot of where that IP is at this moment in time. What's the trigger point for "refreshing" that snapshot? Is it the next report, is it the next update of "seen traffic" is it actually running as a background process, churning away constantly comparing the clock against those ever-changing other variables in the equation?

Share this post


Link to post
Share on other sites

Wazoo, thank you for your reply, but it seems that I am doing a very bad job of trying to get my point accross and as a result your answer misses the point.

I will try again.

I have NO problem with how SpamCop works or with the complex process it uses to determine when and for how long an individual IP gets listed. The fact is I simply accept it and do not even care to fully understand it (that's probably impossible anyway)

In this forum we keep making the statement that if you stop sending spam, then within a max of 48 hours your IP will automaticly be delisted from the SpamCop BL.

My contention remains that is is possible for a IP to remain listed longer than 48 hours after all spam has been stopped. Simple question, is this true or false?

Ellen's statement comes closest to anwering it

It shouldn't be.

16437[/snapback]

But comes short of saying that It will not be. The following portion of your reply might also be interpreted to say that the 48hour rule may be false but that is probably reading more into the reply than was intended.
So, now the catch is ... is there a reporter out there that although reporting the same spam, this user's ISP has a system clock set wrong .... do we have a reporter that noted the date was a bit off and "fixed" it so it would be reported .... do we have a report made with some massively screwed up header lines due to the way the user cut/pasted/forwarded the data .... do we have a condition made famous in the [at]Home days where the e-mail had been sitting on an ISP's server, but that server had just been put back in the rack yesterday and was now merrily delivering that year-old e-mail (doesn't fit the above scenario, just an extreme example)

I simply have a problem with absolute statements that may not be factual.

I can accept a statement that says IP's listed in the SpamCop BL will automaticly be delisted when they stop permitting spam to be sent through them. The delistling will normally occur in 48 hours or less.

I can not accept a statement that clearly states it will happen with in a MAX or 48 hours unless that statement is known to be true.

It has been acknowledged that spam up to 1 week old will be used to determine if, when, and how long a site will be listed and as a result also be delisted.

It has been said that the date and time of reporting does not have any direct affect.

An IP can be manually delisted using the 48hour rule (Yes I acknowledge that there is such a rule) But NOWHERE does it authoratively state that the 48hour rule will automaticly override the complex rule which I still believe is the ONLY rule that works automaticly. The only ones that can answer this question are the programers who actually know how the system really works. The rest of us can only guess.

And yes, I am beating a dead horse, and does it really make any difference, the obvious answer is no.

Share this post


Link to post
Share on other sites
Wazoo, thank you for your reply, but it seems that I am doing a very bad job of trying to get my point accross and as a result your answer misses the point.

I will try again.

I have NO problem with how SpamCop works or with the complex process it uses to determine when and for how long an individual IP gets listed.  The fact is I simply accept it and do not even care to fully understand it (that's probably impossible anyway)

In this forum we keep making the statement that if you stop sending spam, then within a max of 48 hours your IP will automaticly be delisted from the SpamCop BL. 

My contention remains that is is possible for a IP to remain listed longer than 48 hours after all spam has been stopped.  Simple question, is this true or false?

Ellen's statement comes closest to anwering it But comes short of saying that It will not be.  The following portion of your reply might also be interpreted to say that the 48hour rule may be false but that is probably reading more into the reply than was intended.

I simply have a problem with absolute statements that may not be factual.

I can accept a statement that says IP's listed in the SpamCop BL will automaticly be delisted when they stop permitting spam to be sent through them.  The delistling will normally occur in 48 hours or less.

I can not accept a statement that clearly states it will happen with in a MAX or 48 hours unless that statement is known to be true.

It has been acknowledged that spam up to 1 week old will be used to determine if, when, and how long a site will be listed and as a result also be delisted.

It has been said that the date and time of reporting does not have any direct affect.

An IP can be manually delisted using the 48hour rule (Yes I acknowledge that there is such a rule)  But NOWHERE does it authoratively state that the 48hour rule will automaticly override the complex rule which I still believe is the ONLY rule that works automaticly.  The only ones that can answer this question are the programers who actually know how the system really works.  The rest of us can only guess.

And yes, I am beating a dead horse, and does it really make any difference, the obvious answer is no.

16447[/snapback]

Taking the simple case -- i.e. ignoring traffic volumes or the numbe of distinct reporters -- if there are no reports for 48 hours the IP delists. The traffic stats and number of reports (i.e. 2 or less) or reporters may result in a delisting in less than 48 hours. If/when a fresh new spam report(s) is received then the bl logic looks to see whether there has been previous spam still in the "active" state -- i.e. less than 7 days old -- and takes the weighted numbers for those spams and adds them to the weight for the new spam and compares this to the traffic and either relists or doesn't.

Yes if we have the ever famous [at]home situation where random mailservers would magically disgorge old mail with a shiney new received header and/or some other strange thing happens to the topmost received header then the new report(s) may be invalid. That tends to produce email to us and the situation gets rectitfied. If it is user screwing around then they get to become an ex-user. Interestingly enough we do hear from ISPs when they get listed or reports that are wrong -- we even hear from those ISPs that popular opinion states have no sentient life if there is a mistake :-)

I am not sure that this answers your question -- I am not sure that I even understand the question any more but I think I have said everything that I can think of to say on this subject.

Share this post


Link to post
Share on other sites
My question still remains (I know that I am picky) based on Ellen's above statement, can it still be said that there is a true 48hour max clock, or can spam reports that are still being counted for up to 1week cause that the delisting period to extend beyond the 48hour period. I do believe that the 48hour rule is a good general rule of thumb, but I still contend that we are incorrectly stating that all IP's will be delisted within 48hours of the last report.

16409[/snapback]

...IIUC, since the 48 hour clock starts after the last reported spam is received by the reporter's incoming mail server, I believe you are correct to state that "we are incorrectly stating that all IP's will be delisted within 48hours of the last report." It would be more correct to state that "IPs will be delisted within 48 hours of the last reported spam was received by the incoming mail server of the person reporting it." But that might be just too much verbiage and too confusing for the average person who comes here for help and the statement you're disputing is probably sufficient for most cases.

Share this post


Link to post
Share on other sites

Ellen, thank you for your reply. It looks like it does answer my question. But I will restate your answer just to verify that my interpretation is correct.

Part one of the complex formula: If no messages have been reported or spamtraps hit in the last 48hours the IP address will be delisted.

Part two of the complex formula: If part one is false (there has been a report filed or spamtrap hit in the last 48hours) then apply remainder of the complex formula to determine if the IP address should be listed/delisted.

Thank you for taking the time to address this question. It became far more complex than it needed to. Sorry about that.

Thank you again.

Share this post


Link to post
Share on other sites
I can accept a statement that says IP's listed in the SpamCop BL will automaticly be delisted when they stop permitting spam to be sent through them. The delistling will normally occur in 48 hours or less.

IMHO, that's a nice succinct way of stating a very complicated algorithym that depends on a lot of factors that the writers of the algorithym don't really want to explain fully anyway.

I agree that the way the 48 hours often is stated can be misleading to questioners (who are far more interested than we are in /when/). Sometimes IMHO, it is helpful to mention that the time factor is not when the spam is reported, but when it is received that counts.

Miss Betsy

Share this post


Link to post
Share on other sites
Ellen, thank you for your reply.  It looks like it does answer my question. But I will restate your answer just to verify that my interpretation is correct.

Part one of the complex formula: If no messages have been reported or spamtraps hit in the last 48hours the IP address will be delisted.

Part two of the complex formula: If part one is false (there has been a report filed or spamtrap hit in the last 48hours) then apply remainder of the complex formula to determine if the IP address should be listed/delisted.

Thank you for taking the time to address this question.  It became far more complex than it needed to.  Sorry about that.

Thank you again.

16525[/snapback]

If an IP is listed and there are no reports for 48 hours then the IP will be delisted after the 48 hour period. (Let's ignore the 24 hour delists and traffic stats causing earlier delists). If after the 48 hour period one (or more) reports show up then the still "active" spams from the previous week are considered at their current weights in determining if the IP is to be relisted.

That is as clearly as I can state it.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×