Sign in to follow this  
Followers 0
trevorb

New Feature: Greylisting *UPDATED*

81 posts in this topic

We have just added a new spam-blocking feature called greylisting to our mailgates. When enabled, greylisting delays messages from unseen e-mail addresses for a short time (configured to 30 minutes right now). Messages from addresses that have been seen before are allowed through immediately.

"Good" mail relays, like your friendly neighborhood ISP, are set to automatically retry delayed messages periodically, so all of your good e-mail will still get through. Spammers, however, often use relays that don't automatically retry, so a lot of spam will simply never be delivered. This is all transparent to both you and the people e-mailing you, and the only side effect is a short delay the first time someone e-mails you. Our testing over the last few days has shown as much as 75% of spam to a specific e-mail address being rejected by the greylist before it ever hits our spam filters. spam blocked by the greylist is not delivered at all, and will not show up in your Held Mail folder.

More information about greylisting is available at greylisting.org.

We have done some testing and the results have been very positive, but obviously results are not always the same when escalated from a few accounts to thousands. This feature should be considered "beta" for now.

To enable greylisting on your account:

1) Login to webmail

2) Click "Options" on the top menu bar

3) Click "Spamcop Tools"

4) Click "Manage your email forwarding, password, mail report, and greylist settings. "

5) Click the "Enable greylisting" checkbox, and press Submit

Please use this forum to discuss your results. We are interested in hearing about how well this feature works for you. If you encounter any problems, send an e-mail to support[at]spamcop.net.

***UPDATE***

We have added management pages so you can view the messages that are pending in your greylist, and the messages that have been permanently blocked in the past 72 hours. Click Options->Spamcop Tools->Manage Greylist - ... to view your greylist entries. From these pages you can manually unblock senders.

~Trevor

Share this post


Link to post
Share on other sites

Wow, this sounds good.

But I pop down 99% of my mail from other accounts. Won't greylisting

create hassles with my other ISP's ? The only mail I get to my SpamCop

email address is spam from idiots ... or my Held Mail digest, etc. and mail

from the deputies after I screw up something ...

amenex

George Langford

Share this post


Link to post
Share on other sites
But I pop down 99% of my mail from other accounts. Won't greylisting create hassles with my other ISP's ? The only mail I get to my SpamCop email address is spam from idiots ... or my Held Mail digest, etc. and mail from the deputies after I screw up something ...

Any mail that is POPed by our servers does not get greylisted. Enabling greylisting shouldn't affect it.

If you *forward* e-mail to your Spamcop account from another service, it *will* be greylisted, but it will also always be allowed through whether it is spam or not since your ISP is relaying it. If the majority of your mail is forwarded to your Spamcop account, enabling greylisting is probably more harmful than helpful.

-Trevor

Share this post


Link to post
Share on other sites
But I pop down 99% of my mail from other ...

To clarify this for me and all, does that mean you are using POP to move mail off other servers and into this one? That's what I understand it to mean. So you are reading Email on Webmail.

The other method would be to use POP to move Email off here and read it on another Email client like Outlook.

Share this post


Link to post
Share on other sites

To clarify how greylisting affects you based on how you read your e-mail:

If you are using our system to POP mail from your ISP (i.e. move it from your ISP to your Spamcop account), greylisting will do nothing. If you are using POP to download e-mail from Spamcop (i.e. move it from your Spamcop account to your ISP or home machine), greylisting *will* work. Greylisting also works if you are using IMAP to read your e-mail without removing it from the servers, either by using Webmail or your own e-mail client.

Greylisting only benefits you by blocking spam originally sent to your Spamcop account. If the spam was sent to another e-mail account originally and redirected to your Spamcop account (either through ISP forwarding or POP), greylisting does nothing for you.

-Trevor

Share this post


Link to post
Share on other sites
We have just added a new spam-blocking feature called greylisting to our mailgates. When enabled, greylisting delays messages from unseen e-mail addresses for a short time (configured to 30 minutes right now). Messages from addresses that have been seen before are allowed through immediately.

Is the "seen" list initially populated from our whitelist, or do we have to experience the delay once for every one of those, too?

Share this post


Link to post
Share on other sites

Ok, If I enable greylisting, do I in effect not help the SPAMCOP system?

That is, if 'greylisted' mail is never seen, it never can be reported on....

I do 'pop' all of my mail. What I am using this for is bascially 'greylisting' people who send to my [at]spamcop.net address.

Share this post


Link to post
Share on other sites

I use Sneakemail and that service has had greylisting for some time; I love it. However, they also have three views of greylisted mail: pendings, windows, and giveups. All views are useful, but the giveups in particular is VERY useful -- it is the only way to see if legitimate mail was bounced because the sender's servers are misconfigured.

I didn't see an equivalent being offered here, but perhaps I just missed it. How will SpamCop users know if legitimate mail has been rejected due to server that didn't follow the RFCs and make the window?

Thanks and regards,

--appyface

Share this post


Link to post
Share on other sites
Ok, If I enable greylisting, do I in effect not help the SPAMCOP system? ...
I'm interested in the answer to this as well.

I've had the same question about the Held Mail folder, but haven't had time to look for it in the FAQ or on the forum. I've often wondered if I need to report spam in the Held Mail folder, but so far I've decided that if it's held then it still might not be spam, so I report it anyway.

I also get most of my mail by POP'ing it over from my ISP, but spammers are starting to spam me at my spamcop email address. The greylisting will at least help with that.

Share this post


Link to post
Share on other sites
How will SpamCop users know if legitimate mail has been rejected due to server that didn't follow the RFCs and make the window?

This is definitely a question I would like SpamCop to address.

We need more controls over greylist settings. I would want a greylist to automatically use my whitelist to clear some e-mail addresses automatically, and I would want to be able to see and change the e-mail addresses that are cleared or uncleared that are on the greylist.

Share this post


Link to post
Share on other sites
I use Sneakemail and that service has had greylisting for some time; I love it. However, they also have three views of greylisted mail: pendings, windows, and giveups. All views are useful, but the giveups in particular is VERY useful -- it is the only way to see if legitimate mail was bounced because the sender's servers are misconfigured.

I didn't see an equivalent being offered here, but perhaps I just missed it. How will SpamCop users know if legitimate mail has been rejected due to server that didn't follow the RFCs and make the window?

I'm concerned about this, too. There have been times when a user's email gets thrown into Held Mail simply because I chose to remove a full domain from my whitelist, or their email changed slightly (e.g. from my-friend[at]mail.friendlyISP.com to my-friend[at]friendlyISP.com, or even the rarer change of top-level domains), or they simply moved to another ISP.

There are two big advantages to the Spamcop system over others like Earthlink:

  1. Nothing gets lost without my knowing it;
  2. I get to do something about spammers, beyond simply ignoring them.

I fear greylisting removes much of that.

Nonetheless, thanks for offering it, and thanks for making it optional :excl:

Share this post


Link to post
Share on other sites

I fear I'll be switching off the gray list for the time being. I tried it today, and it sadly failed my simplest of tests.

Eight hours ago, I sent an email from a whitelisted address to a spamcop filtered email account and it still hasn't arrived.

I love the idea of the gray filter but until it has a few safeguards to ensure legit mail hasn't been lost or delayed beyond a reasonable time, I'll wait to implement.

JRS

Share this post


Link to post
Share on other sites

There are two major types of SpamCop users:

1) Those who want to see every mail coming in, decide if it is spam or not, and report the spam by hand

2) Those who just want little spam with no input on their side

SpamCop has traditionally focused on the first group, which is why spam is always allowed in and stored in your Held Mail folder. It is not reported until you actually click the "report as spam" button or forward it to a spam reporting address.

Recently, many of our users have been requesting a way to just *block* spam. Greylisting is the first of those methods that we are going to implement. The spam is not reported and doesn't help get spammers shut down or blacklisted... the mail just vanishes. Any system that makes the decision to delete spam without human interaction (i.e. what group 2 wants) *will* result in a tiny percentage of good mail being lost. That is why we have always focused on group 1. But demand is high enough now that we are offering this service to people who want to reduce their spam, and can survive losing a very small amount of good e-mail.

A page to monitor your own greylist entries was considered, and is still on the list of potential improvements. We decided to roll it out now to see how well it works, but a way to manage your own greylist and view statistics is planned.

Currently, the greylist does not consider your SpamCop whitelist. Even addresses on your whitelist will be delayed the first time they are received.

-Trevor

Share this post


Link to post
Share on other sites

I was wondering if the filters are also going to be improved. Currently I need to log-in and manually apply the filters...it would be such a big improvement if the filters could be applied automatically as mail is coming in and/or being retrieved via POP3.

Share this post


Link to post
Share on other sites

[at]trevor

Your description of how greylisting would work is exactly how Sneakemail does it. The difference is, IF I CHOOSE TO DO SO, I can look at the three views in Sneakemail and see what is happening with any mail at each stage of the greylisting process. That approach won't bother your people that don't care what happens to the mail.

And that approach doesn't mean I won't lose legitimate mail, it just means I will be able to KNOW that I lost it. That gives me an opportunity to contact the mail owner and let them know I didn't receive it, and make other arrangements. The sender probably can't contact me, when he/she receives the bounce, because all they have is an email address for me and it isn't going through. I have to be able to contact THEM.

In the case of Sneakemail which uses multiple receiving email addresses, I can simply turn off greylisting for that receiving email address until my legitimate sender's server issues are resolved. A good point raised by UltraJoe is to have the SpamCop greylisting ignore addresses already on the whitelist. This would serve pretty much the same function as being able to selectively turn off the greylisting processing.

The other bells and whistles mentioned such as editing email addresses, etc. would certainly be nice, but if I could just have the same views as Sneakemail and know that a greylisted mail failed (and why) and then whitelist the sender, that would be good enough control for me to use the feature.

Thanks and regards,

--appyface

Share this post


Link to post
Share on other sites

We're getting, roughly, 45% of mails POP'ed down from other accounts, 45% forwarded from other accounts and 10% directly to the Spamcop account's address. Most of these 10% are messages between my partner and I (exclusively) when we're at different locations, as well as the occasional admin mail, but also and worryingly (where did the suckers get the address from?) some spam in the recent past. Usually things gets looked at in the webmail interface, spams reported, unnecessary mails deleted, etc. then real reading, answering and archiving take place in our mail application Entourage, after fetching what's left in the webmail.

If I understand the whole discussion well, we don't need to activate greylisting, especially as we're already losing mails (probably not due to Spamcop) because we correspond in all sorts of languages with people all over the world. Correct, Trevor?

But what we very much would appreciate (and I've asked Spamcop a couple of times already) is to make the whitelist (and the blacklist, while you're at it) more manageable. I've got 18 pages at this point, and I've been through hell a couple of times trying to clean the mess up after clicking (sheer :wub: distraction) on "Release and Whitelist" instead of "Report as spam"

Michel

Share this post


Link to post
Share on other sites
The other method would be to use POP to move Email off here and read it on another Email client like Outlook.
Can't imagine anyone reading email on a client like Outlook. For my part, I use Thunderbird. Which is a GOOD email client.

Not complaining - Just pulling your chain. ;)

But I will definitely give greylisting a shot. Sounds like a useful and desired improvement.

Edited by svanslyck

Share this post


Link to post
Share on other sites
Is the "seen" list initially populated from our whitelist, or do we have to experience the delay once for every one of those, too?

You will experience the delay for those, too. The vast majority of users will not notice or even realize that their mail was delayed. However, if it is important to you that all of your email is received instantly after it is sent, greylisting may not be a great option for you.

JT

Share this post


Link to post
Share on other sites
Ok, If I enable greylisting, do I in effect not help the SPAMCOP system?

That is, if 'greylisted' mail is never seen, it never can be reported on....

I do 'pop' all of my mail. What I am using this for is bascially 'greylisting' people who send to my [at]spamcop.net address.

As time goes on, I think the majority of users don't ever report their spam, they just want it removed. This removes a lot of spam (and viruses) without relying on particular keywords or blacklists. If you want all your spam, though, you shouldn't enable it.

This is definitely a question I would like SpamCop to address.

We need more controls over greylist settings. I would want a greylist to automatically use my whitelist to clear some e-mail addresses automatically, and I would want to be able to see and change the e-mail addresses that are cleared or uncleared that are on the greylist.

We're going to be working to add some additional information. Greylisting should "just work" though. It's really not intended for you to have to go in and fiddle with.

We are working on allowing addresses in your personal whitelist to pass without being delayed. That feature isn't available right now, though.

I fear I'll be switching off the gray list for the time being. I tried it today, and it sadly failed my simplest of tests.

Eight hours ago, I sent an email from a whitelisted address to a spamcop filtered email account and it still hasn't arrived.

I love the idea of the gray filter but until it has a few safeguards to ensure legit mail hasn't been lost or delayed beyond a reasonable time, I'll wait to implement.

That isn't on our side. That's your mail server losing your email. I'd be very interested in looking in the logs to see what happened. Can you email the address that you were emailing from and to to me at support[at]

Thanks

JT

Share this post


Link to post
Share on other sites
We're getting, roughly, 45% of mails POP'ed down from other accounts, 45% forwarded from other accounts and 10% directly to the Spamcop account's address. Most of these 10% are messages between my partner and I (exclusively) when we're at different locations, as well as the occasional admin mail, but also and worryingly (where did the suckers get the address from?) some spam in the recent past. Usually things gets looked at in the webmail interface, spams reported, unnecessary mails deleted, etc. then real reading, answering and archiving take place in our mail application Entourage, after fetching what's left in the webmail.

If I understand the whole discussion well, we don't need to activate greylisting, especially as we're already losing mails (probably not due to Spamcop) because we correspond in all sorts of languages with people all over the world. Correct, Trevor?

Most of the usefulness of greylisting comes from email sent directly to your spamcop.net account. If it is forwarded or we POP it for you, greylisting won't help much.

JT

Share this post


Link to post
Share on other sites

FWIW, I forward all of my email accounts to my SpamCop mail and since I started graylisting, I've seen only 10-15% of the normal spam levels. This could be coincidence, but it's been consistent.

Share this post


Link to post
Share on other sites
Most of the usefulness of greylisting comes from email sent directly to your spamcop.net account. If it is forwarded or we POP it for you, greylisting won't help much.

<i>Won't help at all</i> ITYM so switching it off for trusted relays and other forwarders may save trouble.

I have enabled greylisting and will report.

About 30-40 % of my spam is direct to spamcop mail (rest is POP and forward) so there should be some useful benefit.

It does seem to me that server IP addresses rather than or as well as "From:" should be placed on the good list

This would cut down volume of items to remember and save both forwarded and normal mail being delayed for mail from each new correspondent.

I have also come across a mailing list which used a different "From:" for every item (to keep the threading in order).

Share this post


Link to post
Share on other sites
FWIW, I forward all of my email accounts to my SpamCop mail and since I started graylisting, I've seen only 10-15% of the normal spam levels. This could be coincidence, but it's been consistent.

And I presume you're getting all of your good mail. :)

This is interesting. I'd like to see more data. If this holds up, there is a possible explanation. Email from new, unknown users forwarded by your ISP will all get greylisted and delayed. Your ISP will retry, of course, so all of this spam will eventually be delivered by us. However, during this time interval all of the blacklists that we use have had time to update. Delaying delivery of your spam by 30-60 minutes might make a real difference in how much the blacklists can catch.

If you actually aren't getting the spam at all, either to your inbox or the Held Mail, it might be that a lot of your spam was actually being sent directly to your SpamCop account. Greylisting will help remove a lot of that spam, even if 100% of your legitimate mail is forwarded to us by another ISP.

<i>Won't help at all</i> ITYM so switching it off for trusted relays and other forwarders may save trouble.

Well, no, see my other post about delaying delivery of spam. This is theoretical. I honestly don't know how much difference it makes. I do know that the SpamCop blacklist is very real-time and new spam sources are often detected within minutes.

JT

Share this post


Link to post
Share on other sites

http://www.spamcop.net/sc?id=z1414845532zd...2875ec5dc920dbz

IP source 84.229.49.9

http://www.spamcop.net/sc?id=z1414845542zd...f8f63cbb327ff4z

IP source 218.7.192.70

Turned on GreyListing and leave spam filters active

There are still some getting to my held mail? None seem to be mail servers? Why are they getting through

I only use SpamCop email no forwarding or POP

spam is greatly reduced however

Others

http://www.spamcop.net/sc?id=z1414845551z1...3a6a3abfabd784z

http://www.spamcop.net/sc?id=z1414845559zc...6b7eff90311223z

http://www.spamcop.net/sc?id=z1414845571za...16f3a6da5cb47ez

http://www.spamcop.net/sc?id=z1414845580zd...088ae7057b4b07z

Not sure yet if "innocent' but incompetent providers are getting bounced but it is worth a try

Share this post


Link to post
Share on other sites

Interesting to note ... I turned on greylisting last night, and from that moment on I have received NO spam at all ... I've had nothing in my held mail all day.

I think it's interesting because I use spamcop to POP my mail from my ISP. Then pull it down into Outlook on my home computer.

From what I've read in this forum so far, this shouldn't be the case ... Whatever the case ... I'm doing a test by turning greylisting OFF on just one of my accounts.

I'll let you all know what happens after the next 24 hours!

Very interesting service!

mrcj

BTW ... I wouldn't be without Spamcop ... but I'm also part of that group who likes to report as much spam as I can, so it's entirely possible that greylisting "may not be for me" ... On the otherhand ... I'll never leave Spamcop! You guys ROCK! :wub:

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0