DNS based systems using Spam Cop, Yahoo Groups complaint Options

[Sherlynn]

This is the first I have used this message board, and if I am in the wrong spot, please let me know.

A friend of mine has a problem with her internet server, who is using Spam Cop. She is a member of 14 yahoo groups, and once a month for usually 2-3 days her mail to the groups and from the groups do not get through to her or the groups she is in. Her internet company says it has to do with spam cop and it will work itself out. Well this month, it's been an ongoing problem for the last 3 weeks. She has been in touch with her isp's tech support, and last night they came out and told her there is nothing they can do to fix the problems and she will have to deal with it. I have also been in touch with her isp tech support and this is the response I got to my e-mail....

"Our company uses a system wide setup that is a DNS based list. Unfortunately
we cannot make individual exceptions, or we certainly would. Even though
this is an aggravation for customers that use yahoo groups, we have
determined that the Spam problem would be a much greater problem if we
stopped using Spamcop all together. I will research a little farther to
see if the folks at Spamcop have any other suguesstions for system wide
setups."

My friend is at her wits end. She is ready to leave all of her yahoo groups (which is her lifeline to people because she can't get out of her house). Does anyone know of a solution for her to be able to stay with her yahoo groups, and is there a solution for her internet company to allow yahoo groups without it being considered spam? Any help would be greatly appreciated.

Thank you for your assistance in advance
Sherlynn

[Wazoo]

Well, I had noticed that you'd spent some time in here over the last day or two <g> .. First of all, this is being moved back over into the Help Forum, as it really isn't an issue with a SpamCop filtered e-mail account or even submitting spam complaints via e-mail.

Second, try the Search function (button at top of screen) ... look for "Yahoo Groups" as the search term, look in either Help or All Forums .... You'll find that YahooGroups has a long history of getting various servers onto various BLs. Some ISPs have allowed their users to individually whitelist certain addresses, others have the blocks and filters in place long before any user action could take place. So if the ISP says that can't do it, end of that discussion. To the best of my knowledge, YahooGroup stuff is still accessible via the web interface ... or she could set up another e-mail account elsewhere to have it sent that way, as long as that e-mail provider isn't also fighting the spew from YahooGroup servers <g>

[Wazoo]

And for yet another approach, some third-party software, say something like that found at http://yahoopops.sourceforge.net/ .... which would bypass the ISP's block on those servers.

[DavidT]

She is a member of 14 yahoo groups, and once a month for usually 2-3 days her mail to the groups and from the groups do not get through to her or the groups she is in.

This part doesn't quite make sense. If her ISP is using the SCBL to block messages from known spam sources, why wouldn't her mail to the groups be getting through? The use of blocklists are a "one-way" action...it blocks incoming, not outgoing messages, so you'll need to review this part of your explanation with your friend for futher clarification and examples, because it simply doesn't make any sense.

My friend is at her wits end.  She is ready to leave all of her yahoo groups (which is her lifeline to people because she can't get out of her house).  Does anyone know of a solution for her to be able to stay with her yahoo groups...

I would also suggest that your friend visit those Yahoo groups on the web instead of receiving and interacting with them via email. I belong to about a dozen Yahoo groups, and I simply log in to Yahoo!Groups once a day to peruse and respond to any messages. Perhaps she's not aware of this option?

...and is there a solution for her internet company to allow yahoo groups without it being considered spam? 

It doesn't sound like her current ISP is going to provide her with any "whitelisting" option that would override their blocking actions. However, I just analyzed the headers of a received Yahoo Groups message, checking all of the IP numbers against the current SpamCop BL and they weren't listed.

In fact, I just looked up a whole series of Yahoo Groups servers and they weren't currently blocked by SpamCop, so the ISP might be incorrect. The blocking might very well be due to other factors.

In any case, I'd recommend that she interact with the Yahoo Groups on the web instead....is that possible for her? If she doesn't like all the ads that appear on the website, there are third-party ad blocking options. I use the Firefox blocker with the "Adblock" extension, and I've got it set so that I don't see any ad graphics on the Yahoo site at all.

dt

[DavidT]

And for yet another approach, some third-party software, say something like that found at http://yahoopops.sourceforge.net/ .... which would bypass the ISP's block on those servers.

...but only if she uses a "yahoo.com" email account to subscribe to the groups. People can sign up for the groups using any valid email address, and since most people access their Yahoo email accounts using Yahoo's web interface, the subject of this topic is likely having her messages sent to the POP3 email account supplied by her ISP.

This is a good solution, however, if the "yahoopops" software isn't too hard to install. I think that everyone who signs up for Yahoo Groups also has a "yahoo.com" address, whether they use it or not, and it's easy for the user to log in and have their messages sent to that address instead of to a non-Yahoo address.

dt

[Miss Betsy]

One other thing that she could do is to tell her sad story to Yahoo. Yahoo could do more to prevent their servers from being listed. She could get the people in her groups to petition Yahoo for "blockfree" servers.

Probably Yahoo does nothing, however, because it is possible to log on to yahoo to access the groups so there is no incentive for them to improve the reliability of the email part.

Miss Betsy

[Merlyn]

A lot of the reliability of the email in Yahoo Groups is upon the Groups administrators. Some administrators allow anyone to join and send whatever they want. Some spammers start groups and add lots of email addresses then spam knowing the group will get terminated. Some Administrators add people manually because they think they want their info. There are many other problems with Yahoo groups but we need not get into them here. It's a blend of bad Yahoo administration and Group Administrators that have no idea how to administer/maintain a group.

While the current situation exists they will ocasionally be on the blocklist.

[Webber]

I'm having what appears to be the same problem as Sherlynn.
After years subscribing to various Yahoo Groups, all of a sudden
the email from my YahooGroups is *intermittently* blocked.

Here's what's known so far...
Yahoo account page called "Email Preferences" shows that
email to my account is on status "Soft Bouncing" (should be
"Normal" status. Yahoo gives reason (in "Bounce History"
section) as follows:

Remote host said: 550 5.0.0 Spam blocked see
http://spamcop.net/bl.shtml?66.94.237.47

That url at SpamCop displays message saying
that IP is NOT on the blacklist. It is NOT being blocked
by SpamCop.

So, that means that at least it SHOULD NOT be being
blocked by SpamCop.

That means (to me) that either YahooGroups is not
displaying an accurate error reason OR that IP 66.94.237.47
IS actually appearing (on and off) the SpamCop blacklist.

The problem goes away temporarily if I reset my Yahoo
Email Preferences to Normal status. The fact that the
problem keeps recurring seems to indicate that something
at Yahoo is whacky, but the possibility that something at
SpamCop intermittently blocks my IP also exists.

I've just written to Yahoo Groups about it and now am posting
here because it might be helpful and because it's possible
some software glitch at SpamCop does intermittently
blacklist IP 66.94.237.47.

I did a search on SpamCop for others who might be
experiencing the problem and have seen quite a few
but none yet which have isolated the source of the
problem and know for certain that the origin is at Yahoo.

Any feedback would be welcome. Thanks.

[StevenUnderwood]

it's possible some software glitch at SpamCop does intermittently blacklist IP 66.94.237.47.

Yes, that software "glitch" is called spam reports and/or spamtrap hits and is the normal way things work at spamcop. Probably, the IP has timed off the list by the time you looked it up but was listed previously.

I have personally (though not recently) received spam from yahoogroups though I have never signed up for any. People who use YahooGroups are sharing IP's with spammers, same as if they were using a spammy ISP.

Senderbase stats for that IP http://www.senderbase.org/?searchBy=ipaddr...ng=66.94.237.47 show more than a 1000% increase in traffic over the 30 day average for that IP and more that 2500% over the last days average.

Now that IP was first seen on 8/16/2004 so it is probably a fairly new IP YahooGroups has implemented. Newness counts in the blocking as well.

This post has been edited by StevenUnderwood: Aug 28 2004, 10:09 AM

[Wazoo]

Problem 1 is that the "evidence" page is no longer real-time, and the content has really been diminished (a result of the apparent use of the data once there to run the ragged edge of listing IPs)

Problem 2 is that the BL listing is based on a mathematical formula, the results of which can have an IP listed for the minimum of a half-hour up to the max of 48 hours after the spew stops.

Problem 3 is the user decision on some of the complaints. On one hand, it's the group manager that should get the first hit, then Yahoo ... but again, the history of some of the YahooGroups uses has caused many a reporter to simply run it through the SpamCop parser, as the previous direct contacts seem to be useless so often. And one can't discount the issue of some users running in blind auto-pilot mode and reporting anything and everything.

[DavidT]

Here's what I think I know about this issue, so far....

1. The Bounces information at Yahoo seems to only be updated weekly, so is often quite out of date, and furthermore, only presents the SMTP error totally out of context, without reference to which server is doing the rejecting.

2. I think that in most of these cases, the rejection is happening at the servers of third-party ISPs who are using the SpamCop DNSBL as part of their spam control strategy, and that perhaps various IPs from the MANY used by the Yahoo Groups servers are making brief appearances on the SCBL, but then falling off after 48 hours.

3. I think that the reason that these specific IPs are appearing on the SCBL in the first place is mostly due to SpamCop users who are mistakenly reporiting mail coming from their own subscriptions to various Yahoo Groups as spam. I've run parsing checks on some of the many IPs, then gone in to look at the reporting "History" for those IPs...here are the Subjects of some of the items that SC users have reported:

Welcome to the new Yahoo! Messenger - check out the great features!
MODERATE -- nirvanawomanmagazine[at]yahoo.com posted to ATML-Film
[kewlgayottawa] I'm gay Jayme's and this is my life.
[Vintage Honda Motorcycles] Free Biker Dating Site
[momsworkingathome] Moms@ Home Working Telecommuting Jobs! (IMG:style_emoticons/default/smile.gif))
[Dignity Discussions] My Parents Are Allies
[SavageNation] Digest Number 339
[WindowsHelpPT] Oportunidade: Analista de Suporte
[Reaktor-list] WOODSTOCK 69 LIVE RECORDING

In all likelihood, NONE of these should have been reported as spam. The "MODERATE" message above was reported by one of the moderators for a "Discussion list for people interested in the on camera or on tape industry in the Atlanta, Georgia area." It was probably in their Held Mail and they blindly reported. All of those messages with the group names tagged at the begging were probably reported by SC users who *subscribe* to those specific Yahoo Groups, so they shouldn't report *anything* coming from the group as spam....period.

Therefore, unless the SC admins can exempt all of the hundreds of Yahoo servers, with names and numbers patterned like this:

n8a.bulk.scd.yahoo.com [66.94.237.42]
n28.grp.scd.yahoo.com [66.218.66.84]
n11.grp.scd.yahoo.com [66.218.66.66]
etc. ad infinitum

from getting on the SCBL, there will be occasional problems.

However, I have a simple solution. If you want to make sure that you receive ALL of the messages sent from the Yahoo groups, all you need to do is this:

1. if you don't already have one, purchase a SpamCop email account
2. log into Yahoo!Groups and add your "spamcop.net" email address as one of the addresses used with Yahoo!Groups (found in "My Groups / Email Preferences")
3. change your subscriptions to each group so that mail will be sent to your SpamCop address
4. log into "https://webmail.spamcop.net" add the following to your whitelisting:

returns.groups.yahoo.com

That will most likely entirely solve the problem, because the third party ISPs won't be handling the messages, and if any of the Yahoo IPs are temporarily in the SCBL, the messages will still reach you because the personal whitelist takes precedence over the blacklists when mail arrives at the SpamCop servers addressed to you. The whitelist is in "Options / Mail Mangement / SpamCop Tools."

Of course, if you don't want to buy a SpamCop email account, the other solutions would be to either try out the "Yahoopops" software that Wazoo mentioned, or to have your Yahoo Groups mail sent to an address at an ISP that allows you to do the same kind of "whitelisting" that SpamCop accounts do. But I think that having a SpamCop email address is the best solution (as long as they fix the "Held Mail is Not Expiring" problem mentioned here!).

DT

This post has been edited by DavidT: Aug 28 2004, 11:21 AM

[StevenUnderwood]

David, I agree with everything you say except you seem to ignore the possibility of people being added to groups without their knowledge or consent. It hs happened to me, though not within the last year. I know Yahoo has done some things to minimize this possiblity, but am not convinced it is solved.

[DavidT]

David, I agree with everything you say except you seem to ignore the possibility of people being added to groups without their knowledge or consent.

I think that Yahoo has indeed addressed this, so maybe it's a moot point. In any case, the evidence seems to show that a LOT of SpamCop users aren't being very careful in their reporting.

dt

[Wazoo]

DavidT's post added as a FAQ entry point ... will bring folks into the middle of the Topic, allowing them to scroll up/down to catch the rest of the discussion if they so choose.

[Webber]

I know I'm missing something but I don't know what it is.

As a reseller of websites for ISPs, some of whom want to use SpamCop's services, it's extremely unpleasant to think that SpamCop has no workaround for the problem... software being so, er, inherently programmable.

What I think I am reading in the thread is that the IP 66.94.237.47, owned by Yahoo, is temporarily added to the SpamCop blacklist if and when one SC subscriber gets the notion he's been spammed by someone using IP 66.94.237.47 and reports it to SpamCop.

Is that correct? One spam report, real or imagined, automatically plunks the IP on the SpamCop blacklist?!?

Thanks.

[DavidT]

What I think I am reading in the thread is that the IP 66.94.237.47, owned by Yahoo, is temporarily added to the SpamCop blacklist if and when one SC subscriber gets the notion he's been spammed by someone
using IP 66.94.237.47 and reports it to SpamCop. Is that correct?  One spam report, real or imagined, automatically plunks the IP on the SpamCop blacklist?!?

No, that's not correct, and I'm sorry if I implied that in my response. It takes more than one complaint to add an email source to the SpamCop BL. Here's an official description of "What is on the list?":

http://www.spamcop.net/fom-serve/cache/297.html

But even if a given IP gets added to the SCBL in error, or unfairly, SpamCop isn't doing any blocking. It's the many ISPs who choose to use the SCBL listings as red flags and who configure their mail servers to outright reject any traffic from listed IPs that are going a bit overboard, IMO. They should be using the BL as a way to "tag" possible spam, perhaps putting it into a temporary holding area that's accessible to the end user for review and processing. They should be offering the end user the ability to "whitelist" not just the individual "From" or source IP addresses, but entire systems, such as "returns.groups.yahoo.com" found in the headers, which is what SpamCop does for its email customers...that's why I recommend that people buy and use SpamCop email accounts, because we are given both powerful filtering, and flexible configuration options.

dt

This post has been edited by DavidT: Aug 28 2004, 12:52 PM

[Webber]

Thanks David. No, you didn't imply that it only required one spam report. I just couldn't eliminate that as a possibility. Thanks for the url describing the criteria SpamCop uses.

It still seems screwy, though. The url text states that: "The reports about SUBE from a given system are weighted against a sampling of the total amount of mail from the same system to determine a ratio..."

I would think that surely, since it's a Yahoo IP, used for umpteen legitimate group mail distributions, the overwhelming number of email messages sent from that IP could be determined to be legitimate and not spam. The "sampling" does not seem to provide a very accurate sampling, if you see what I mean.

Since the IP (66.94.237.47) is relatively new, I'm wondering how that fact (the newness) has perhaps provided a faulty sampling.

Yes, I knew that it wasn't SpamCop doing any of the actual blocking and it's the ISPs who want to use SpamCops blacklist and the ISPs who do the blocking. Here's the problem, though... SpamCop's blacklist and other services offer very fine advantages for ISPs and their users in many, many ways. No doubt about it. However, one enormous disadvantage for resellers of hosting and ISP services (and for ISPs themselves) is requiring clients to find funky workarounds and purchase their own individual SpamCop workaround software just to circumvent what appears to be faulty sampling or whatever it is that's causing a standard ol' Yahoo IP Group Mail server to appear on the SpamCop blacklist. I'm basing this on the notion that even though IP 66.94.237.47 is relatively new, the majority of the mail going through would seem to be overwhelmingly gazillions of legitimate YahooGroup email messages... not "sampled" very accurately, no?

Thanks.

[dbiel]

However, one enormous
disadvantage for resellers of hosting and ISP services
(and for ISPs themselves) is requiring clients to find funky
workarounds and purchase their own individual SpamCop
workaround software just to circumvent what appears to
be faulty sampling or whatever it is that's causing a
standard ol' Yahoo IP Group Mail server to appear on the
SpamCop blacklist.

I am afraid that you are still missing the point.
The workaround software is only necessary because the ISP has failed to provide a proper implemention of the SpamCop blocking list.
It was never intended as a block and reject list which unfortunately is how many ISP's are using it. It was meant to be used as a flagging list to separate "suspect mail" (an Earthlink term) from regular mail.
Earthlink has built its own SpamBlocker with one major benefit over the SpamCop version (unfortunately they do not offer the flexibilty of adjusting the list other than whitelisting, which makes SpamCops version much better overall)
Earthlink's advantage is a two part heldmail folder. One for known spam (near zero false positives) and one for suspect mail. They allow their users to access both files to check for false positives and to do automatic whitelisting of anything so found. This allows users to totally ignore the known spam files if they choose as most users find it not worth the effort to check unless they have an taste for spam.

So back to the main point. If ISP would use the list correctly, there would be no need for any workarounds.

[WB8TYW]

I am in a Yahoo Group that was being spammed once to twice a week by until the moderator got back from vaction and locked the group down to require human approval of new members.

It was apparent from the way that the spammer was operating that they had managed to come up with a system that created Yahoo accounts automatically, and then had those accounts join up as many groups as they could.

It also appears that Yahoo allows a spammer to create new groups, and spam 50 addresses for each group. Since there are no known limits to the number of groups that a spammer could create or have created, there is the possibility of a lot of spam being sent. (They used to allow 100 spams, but cut it back to be less attractive to spammers.)

These complaints seem to go in cycles, as if Yahoo lets the spam level get up to the point that their servers start getting listed, and then do a house cleaning, and that only seems to last about 4 to 6 months.

After the group I was in was locked down, another spam run caused the user list to get audited. Three more spammer accounts where found on it that had not yet made a spam run, indicating the the spammer was creating the accounts a while in advance of the spam run.

And even today, I am getting spam that was relayed through the Yahoo mail servers, in the case of that spam, Spamcop notified the true address, and also notified Yahoo as a relay. But the blocking would apply to the real sender's I.P. address.

If Yahoo is putting the proper headers on the spam, then Spamcop.net should be identifying the true source.



Now as far a tagging and quarantining spam, IMHO, that is a poor way to do it, and is virtually guaranteed to cause real e-mail to get lost with our the sender or the receiver knowing.

See the pinned topic in the lounge on the Cost of Spam.

If an e-mail is known to be undeliverable because it is confirmed spam, then it should be rejected with a 5xx SMTP code at the mail server. No need to quarantine it.

In fact an error where a real e-mail is quarantineed is worse than having it rejected, because then the sender does not know that there was a problem, until the receiver checks their quarantine. And if you have to check the quarantine, then there is no point in using it.


If a mail source is listed in the xbl-sbl.spamhaus.org, or in an open proxy or open relay database, then there is so little chance of it being a real mail, the mail server should just reject it.

This will remove over 80% of the spam with zero false positives reported by many users.

For the remaining:

A listing in the spamcop.net, indicates over an 90% chance of an item being spam.

A bad rDNS listing indicates over an 80% chance of an item being spam.

A bad rDNS listing and a spamcop.net listing indicate an overwhelming indication of spam, no need to accept or quarantine.

A listing in a DHCP pool indicates an over 90% chance of spam, and usually real mail servers operating on DHCP classified addresses have bad rDNS entries.

A listing in a DHCP pool and Spamcop.net is an overwhelming indication of spam, no need for the mail server to accept or quarantine.

This will eliminate almost all the rest of the spam, again with out the spam filter even looking at the content of the spam, so the rejection can be done early in the SMTP dialog.

Some mail server programs can not do scoring or support a mail server, so they must accept or reject solely on the say of a DNSbl. Some of these mail servers operators have determined that the DHCP lists and Spamcop.net are accurate enough.

Others will reject e-mail from spamcop.net listed I.P. with a 4xx code which will cause the sending mail server to retry for up to a week. Usually a real mail server will age off of the spamcop.net blocking list by that time.


To further safely eliminate spam with out impacting real e-mail, if an item is from an I.P. address with a bad rDNS, listed in an agressive list like spamcop.net, or in a DHCP pool, then SpamAssasin 3.0b has a feature that will positively and accurately detect most of the spam that gets through.

What it does is look up the I.P. address that the URLs in the e-mail resolve to, and if any of those I.P. addresses are in any of the DNSbls like the sbl-xbl.spamhaus.org, or spews.org, then the e-mail is highly likely to be spam, and can be rejected.


Any spam filtering system that does not use SMTP rejects on what is determine to be spam is going to have an undetected error rate.

It seems though that many people prefer to have a high number of undetected errors, and undetected lost e-mail, instead of having a very low number of detected errors.


I see the most complaints about lost e-mail from mail servers that accept all e-mail, and then use content filtering afterwords to sort the spam from real mail, and then tag, or quarantine it. I also see the most complaints about spam leaking through those type of systems.

So from my viewpoint, such systems are more complex, more expensive, and less effective than the simpler technique of using SMTP rejects on what has been determined to be spam.

Some spamcop.net users of such systems that tag or quarantine are also the source of causing many real mail servers to be listed.

What happens is that for some glitch, a real mail server gets listed, and enough reporters report everything that is quarantined with out inspection, so that the mail server continues to be listed even after the original problem was fixed.

Blocking lists work because ISPs do not seem to care about fixing problems on their networks until their paying users start complaining to them that the rest of the internet is no longer accepting e-mail from them.

-John
Personal Opinion Only

[SpamCopAdmin]

The Yahoo Groups servers send a lot of spam. As a result, they go on and off our list as the amount of spam ebbs and flows. Service providers who use our service will refuse to accept mail from Yahoo servers on our list, which causes their mail to some users to bounce, which causes the subscription to be suspended in turn.

Many of the Yahoo Groups mailing lists are owned by spammers, and by clueless list owners who think it's OK to harvest email addresses from the web, and from Usenet newsgroups, and add them to their list without permission.

There is no doubt the traffic is spam, either, because a lot of it comes to our spamtraps. Mail to nonexistent addresses is proof-positive that email addresses are being added to a mailing list without the address owner's permission.

That's not to say there are no user errors because there are. Especially by SpamCop Filter users who report the traffic when it's diverted to their spam folder because it came from a blocked Yahoo server. All possibilites apply.

- Don -