Jump to content

Yahoo! groups


F. Jones

Recommended Posts

But how could someone know if your users asked for a specific list?

The issue is much larger than what my specific users want right now. The point is, the SpamCop list, in its current state, has the potential to block legit traffic. Quite a lot of it. An unacceptable amount, in our case. Since I know this, I'm not using the list, but if the list didn't include all these known sources of legit traffic that are sometimes blocked (or at least as many as could be reasonably expected with a reasonable method of adding/deleting entries as necessary), I would use the list, and so would just about all the other admins in my situation.

And anytime you open a hole in a blacklist, you open the possibility for spam to come through.

Yes, I completely understand this.

I know you are more worried about not blocking something requested, so perhaps you need to create and use only your own blocklist.

I use quite a few BLs, all of them much more conservative and accurate than SpamCop, when it comes to false positives. This isn't a slam on the SpamCop system, which works very well in many cases. It's just that I have no use for the system in its current state, which should not be a controversial statement, since the SpamCop web site says the very same thing.

IMHO, those who use the internet need to be aware of basic internet concepts. And blocking is one of them. If your users want to access Yahoo groups, let them complain to Yahoo for being irresponsible or help them deal with the spam.

It's nice that you all have these very strong opinions about laying down the law and whatnot, but it's not realistic to assume that all organizations will agree with them.

Link to comment
Share on other sites

QUOTE (Miss Betsy)

IMHO, those who use the internet need to be aware of basic internet concepts. And blocking is one of them. If your users want to access Yahoo groups, let them complain to Yahoo for being irresponsible or help them deal with the spam.

(F. Jones)It's nice that you all have these very strong opinions about laying down the law and whatnot, but it's not realistic to assume that all organizations will agree with them.

You are specifically asking about the use of the spamcop blocklist. As you point out, other blocklists are more conservative and can be used. The unfortunate part of that is that the end users don't know that blocklists are the most effective way to block spam (and in the end control it for most users). If end users knew that there *was* something that can be done about spam and that they could participate in it, then many more end users would demand that ISPs use blocklists and be willing to explain to correspondents who were blocked that the *sender* of email needs to be responsible for choosing a reliable ISP.

I admit that many technically non-fluent people who post do not seem capable of understanding basic concepts about email. However, there are enough competent, non-technical people who use email who can understand how blocking works and why it is effective that if ISPs enlisted their help and support, the 'big boys' like yahoo and comcast would soon make it very difficult for spammers to use their services.

They might even be vocal enough that other blocklists would be created that are effective as spamcop, but not as aggressive.

IMHO, many of the organizations that don't agree just want to pretend that they are superior to the end user and know better than the end user what the end user wants. Sort of a 'nanny' complex.

Other businesses don't want to lose any leads that might result in a sale in which case they don't use any blocklists to reject which I understand. They, however, have developed better ways to filter, but don't contribute much to the control of spam. In fact, those who create spam content filters have no stake in stopping spammers.

I forget the original purpose of this topic. Wasn't it something about whitelisting yahoo groups? If one does that, then one needs to 'help the users deal with the spam' through content filters. It is either one or the other; blocklist and educate end users why this is a good thing <or> don't blocklist and teach them how to use content filters.

Miss Betsy

Link to comment
Share on other sites

The issue is much larger than what my specific users want right now. The point is, the SpamCop list, in its current state, has the potential to block legit traffic. Quite a lot of it. An unacceptable amount, in our case.

This is the same for almost every dnsbl out there. If legit traffic MUST get through, then NO filtering should be done. It is a balancing act between the pain of spam and the pain of missing a message.

Since I know this, I'm not using the list, but if the list didn't include all these known sources of legit traffic that are sometimes blocked (or at least as many as could be reasonably expected with a reasonable method of adding/deleting entries as necessary), I would use the list, and so would just about all the other admins in my situation.

So you are are looking for a list of servers, that while they send some spam we'll ignore that because they also send legitimate messages. Well, that goes for most servers on any blocklist you will find. The only lists I can think of that would not list alot of valid mail servers would be the open proxy or virus infected lists. Perhaps you should use some of those.

I am not telling you that the spamcop list is good for everyone. I don't use it at work myself because my administration wants to be able to access the held messages, so we use postini services to do the filtering for us, and they have worked out extremely well. It does as it is defined to do, to quickly block current sources of spam, usually during the spam run itself. Personally, I don't like the spamtrap setup as it seems to be listing too many misconfigured servers as opposed to spamming servers.

Link to comment
Share on other sites

I admit that many technically non-fluent people who post do not seem capable of understanding basic concepts about email.

A lot of my users would be baffled by the concept of an online forum and would be frustrated by the thing long before they figured out how to post. I'm just saying...there are people using the Internet (some of whom are otherwise brilliant, much smarter than you or I) who just don't get Net culture. The sort of people who will never figure out what a file is or how the directory structure of a hard drive works, no matter how much time you spend at the whiteboard.

Hanging around online forums, we can lose sight of who we're talking about when we discuss "users".

They might even be vocal enough that other blocklists would be created that are effective as spamcop, but not as aggressive.

There are actually a few of these, but no matter how good a list is, there's always going to be something that one list gets that another doesn't.

The vast majority of my spam is being blocked...I'm just always looking for new lists to tackle that last little bit of spam. SpamCop would only take care of about 10% of the spam that's still trickling through at my site, but even one less message a day is an improvement.

...organizations...pretend that they are superior to the end user...'nanny' complex...don't want to lose any leads that might result in a sale...

These are all fine theories, but they don't apply to my site.

I forget the original purpose of this topic.  Wasn't it something about whitelisting yahoo groups?

This thread, yes. I posted another, related-but-separate, question in a new thread, because I wanted to talk about more than Yahoo! groups (ie, whitelists in general), but The Powers That Be decided that that post should be moved into to this thread.

If one does that, then one needs to 'help the users deal with the spam' through content filters.  It is either one or the other...

I don't agree with that, because...

This is the same for almost every dnsbl out there. If legit traffic MUST get through, then NO filtering should be done.

...while this is true, every other list I subscribe to manages to not block Yahoo! groups, and other known sources of spam that also send legit traffic. I talked to some people and it seems that they maintain whitelists, but for various reasons don't want to share them.

Perhaps that's the path I should proceed down.

So you are are looking for a list of servers, that while they send some spam we'll ignore that because they also send legitimate messages. Well, that goes for most servers on any blocklist you will find.

Actually, most of the conservative lists seem to have a built-in whitelist to avoid this problem.

The only lists I can think of that would not list alot of valid mail servers would be the open proxy or virus infected lists. Perhaps you should use some of those.

I do, thanks.

Link to comment
Share on other sites

I talked to some people and it seems that they maintain whitelists, but for various reasons don't want to share them.

I have never heard of anyone making a public whitelist. As I have said before, your decision on what to let through should not be based on what others let through, but by the tendencies of your users and what they want to receive. If I made public the whitelist my company uses, it would reveal all of our corporate partners, some of whom we do not want to be made public to the competition.

Actually, most of the conservative lists seem to have a built-in whitelist to avoid this problem.

Spamcop has never claimed to be a conservative list. To the contrary, it claims to fault to the agressive side. It also drops entries automatically when the spam reports stop. If you are down to 10%, that might be as good as you can get using the conservative lists. That is, obviously, your decision.

I do, thanks.

You are welcome, good luck.

Link to comment
Share on other sites

A lot of my users would be baffled by the concept of an online forum and would be frustrated by the thing long before they figured out how to post. I'm just saying...there are people using the Internet (some of whom are otherwise brilliant, much smarter than you or I) who just don't get Net culture. The sort of people who will never figure out what a file is or how the directory structure of a hard drive works, no matter how much time you spend at the whiteboard.

Brilliant people manage to drive cars without being reckless or knowing very much about how a car works (though I did know someone who threw away a tire because it was flat). And if brilliant people are fleeced by an irresponsible ISP, that happens offline also - though, usually if they are really intelligent, they have ways of finding responsible mechanics.

And I wasn't talking about online forums, but ISPs educating their users. Educated consumers no longer look at their doctor as a god, but get second opinions and there are many other areas where consumers have been able to change the way an industry does business because they were interested. It would not take a good pr person very long to convince email users that blocklists are the greatest thing since sliced bread.

And you don't have to know about the directory structure of a hard drive or even what a file is. All you have to know is that your ISP can stop spam from entering your inbox most effectively by blocking it and that also makes you, and your ISP, good, responsible netizens. Anyone who gets blocked is either irresponsible or has been caught in the equivalent of a traffic jam because of irresponsible users or one of those glitches of modern life that we all endure daily.

I am not technically fluent. I don't use half the features on this forum, because I don't have time to figure them out. I don't change the oil in my car, either. But, I do know that my mechanic is not supposed to dump the old oil down the sewer.

Miss Betsy

Link to comment
Share on other sites

As I have said before, your decision on what to let through should not be based on what others let through...

Somehow the conservative blocklists manage effective whitelists that are not based on my specific needs.

Spamcop has never claimed to be a conservative list.  To the contrary, it claims to fault to the agressive side.

The real shame here is that it'd be fairly easy (relative to other efforts on this scale) to make SpamCop into a more usable list. (It could be as simple as making a deal with another well-respected list, although from what I've read, that doesn't sound likely from a political standpoint.) I guess the list itself is still experimental, so this will probably be the next step in the evolution of SpamCop...a functional list for those who only want to block and are not anti-spam zealots (in addition to the current list, not in place of).

If you are down to 10%, that might be as good as you can get using the conservative lists.

The 10% I mentioned was how much of the spam that's still making it thru would be filtered out by SpamCop. I'm actually accepting much less than 10% of all the spam directed at my server, which is pretty good for straight blocking, with no content-scanning.

Obviously I'll never get too close to 0% with this method, but given the trade-offs I have to live with, this isn't bad at all.

Link to comment
Share on other sites

Somehow the conservative blocklists manage effective whitelists that are not based on my specific needs.

Which lists are using a whitelist? I would be interested in looking into that. This is the first I've heard about that, though to be fair, I curently don't use bl's directly, and have not researched them thoroughly. We are looking into that possibility, however.

Link to comment
Share on other sites

And I wasn't talking about online forums, but ISPs educating their users.

In this case, I am the ISP for these users, and let me tell you, they are never going to achieve the level of technical expertise you (or I) want them to. Some of these people are using 20-year old e-mail clients and just can't grasp the concept of a GUI or (for conceptual or physical reasons) cannot use a mouse.

Anyway, the point I was making is that I realize I'm at a hive of activism here and in general I support that kind of thing, but it's easy for people in this atmosphere to lose sight of the Big Picture.

Link to comment
Share on other sites

Which lists are using a whitelist?

I know this one does, for example. I can only assume that some of the larger lists do as well, based on what they block.

I can see if a list is overly aggressive within just a few minutes of applying the filter, since they'll start blocking Yahoo! groups right away. There would certainly be other false positives, but those are the most apparent, and easy to detect, since I can turn the filter on at 4am and see results almost immediately with the lowest probability of blocking a standard person-to-person communication. (I don't really have the luxury of waiting until a bunch of real stuff gets blocked, so I have to move tentatively.)

Anecdotally I've seen discussions of IPs that are not "safe" to block, and since those never, ever get blocked using the lists I do, I can only assume that they're whitelisted.

Link to comment
Share on other sites

...expertise you (or I) want them to.

I keep telling you *I don't have the expertise either* I don't know what a GUI is off the top of my head and I used the mouse for years just to move the cursor around and never used any of the other functions. I still don't what happens if you click those top two buttons.

It is not a matter of teaching them *how* it works. It is a matter of convincing them that blocklists are a GoodThing and to tell their correspondents who get blocked that they are the ones at fault (or that "Isn't awful the way computers always act up when it is important?).

It is obvious that if everyone whitelists the big senders, that the big senders have no incentive to change the way they do things and that everyone else has to clean up after them. It is also true that offline, that consumers have pushed *responsibility* on large corporations by the fact of their opinions.

The concepts of the Internet and of blocking are not beyond the capability of the average user to understand. All things come in bell curves and there will always be the equivalent of the idiot who threw away the tire as well as users who can tell you exactly what the problem is. There aren't many people who can tear an engine apart and put it back together again. And there are lots of people who refer to 'funny noises' under the hood who are still capable of choosing a mechanic and know some things to look for.

Miss Betsy

Link to comment
Share on other sites

It is obvious that if everyone whitelists the big senders, that the big senders have no incentive to change the way they do things and that everyone else has to clean up after them.

This might interest my users on a purely theoretical level, but given the other options available out there, it would not convince them that using SpamCop is a good idea (as a pure blocklist).

The point I'm trying to make is that in many ways this forum exists outside the bounds of what is considered reasonable and necessary in other circles. As an anti-spammer brother-in-arms, I agree with the philosphical points you're making, but that doesn't change my obligations to configure my network in a useful manner.

What I'm trying to say is...not all admins are allowed to be activists, and a little flexibility on this side of the issue would be welcome.

Link to comment
Share on other sites

Not sure why you keep trying to keep this discussion going utill it meets your terms or your definition of how to block or until someone agrees with you. Many of us are admins and fight spam in different ways. The blocklists used are the choice of the admins. Spamcop is a list just like the other blocklists. With over 450 blocklists available you make the decision on which to use. If you do not like Spamcop then don't use it, if you don't like the CBL, don't use it, if you don't like Spamhaus don't use it (I won't do this 450 times.) Whatever works for you might not work for someone else and whatever works for them might not work for you. This is not philosophical and owners of blockllists do not change their criteria just because it "might" be right or wrong. Admins that use Spamcop to block/tag spam use it because it works for them. This is a very simple process and you have a lot of flexibility. Use other lists.

How long can you beat a dead dog to death?

If this thread keeps going it should be moved to the lounge as it has nothing to do with Spamcop help anymore.

Link to comment
Share on other sites

<snip>

I can see if a list is overly aggressive within just a few minutes of applying the filter, since they'll start blocking Yahoo! groups right away.  There would certainly be other false positives, ....

<snip>

...What is your definition of a "false positive?" Mine is something along the lines of "identifying some entity (in this case, an IP address) as something it is not (in this case, a spam source)." Using this definition, I would not consider the identification of IP addresses through which e-mail from Yahoo! groups is routed as a spam source to be a false positive, since (IIUC) they are a spam source.

Link to comment
Share on other sites

a 'false positive' in this context is an email that is not a spam email that comes from the same IP address as spam.

There are many people who do not want to incur any 'collateral damage' (IMHO, in the mistaken belief that people use a spammy ISP are 'innocent' just as those who become infected by viruses are 'innocent' victims. I prefer to say that they are 'ignorant' and that 'ignorance is no excuse')

F. Jones does not think that he can explain to his users who are members of yahoo groups why they cannot receive their groups any more. In fact, he is not even going to make an attempt. IIRC, it may be that his users are members of yahoo groups for other than recreational reasons so that the logic is similar to businesses who do not want to block anyone who might be a customer. But a major part of his reason is that even brilliant people in other fields can't understand the concept of blocking/rejecting as the most effective way to control spam and the concept that the *sender* is the only one who can effectively stop spam.

F. Jones likes the spamcop bl except when it blocks 'false positives' or the poor 'innocents' who can't understand enough about the Internet to be responsible users - particularly those on large ISP's (like yahoo and probably Comcast). He thinks that other admins must have the same concerns and would have a whitelist of large/popular ISP's who are irresponsibly allowing spam to come from their systems on a regular enough basis to get listed on spamcop. But because this is the spamcop forum and therefore a "hive of activism" no one has such a whitelist for him to use (besides not even comprehending why he would want such a list).

He is not disputing any of the principle (or theory) that has been propounded in answers to his posts; he just doesn't think that it is practical in today's marketplace. And he is correct that most admins agree with him. Which is why spam is still continuing to grow in percentage of total emails and F. Jones is looking for ways to filter it out.

If F. Jones is still reading this, perhaps he will say whether I have summed it up correctly. Of course, he wouldn't phrase it quite the way I have. He would say that his approach is like politics 'the art of the possible' (based on ideals, but tailored to fit what can actually be accomplished in the real world).

Miss Betsy

PS As Merlyn says, this thread is essentially 'dead' If someone wants to start another one or continue the discussion of theory vs reality, it really should be moved to the Lounge

Link to comment
Share on other sites

I'm happy to consider this thread dead as well, but I do want to clear up Miss Betsy's interpretation of what I've said...

F. Jones does not think that he can explain to his users who are members of yahoo groups why they cannot receive their groups any more.

I could certainly explain this, and they would understand, and then I would be told to undo what I'd done, because a higher priority has been placed on users receiving their mail than crusades, regardless of their value.

But a major part of his reason is that even brilliant people in other fields can't understand the concept of blocking/rejecting as the most effective way to control spam and the concept that the *sender* is the only one who can effectively stop spam.

As I've explained before, that's not the reason I brought up the lack of technical expertise.

He thinks that other admins must have the same concerns and would have a whitelist of large/popular ISP's who are irresponsibly allowing spam to come from their systems on a regular enough basis to get listed on spamcop.

I don't "think" this, I know this to be true, based on what other lists admins in my position are using, and the fact that they employ whitelists.

But because this is the spamcop forum and therefore a "hive of activism" no one has such a whitelist for him to use (besides not even comprehending why he would want such a list).

I'm not the one in this thread with a comprehension issue.

He is not disputing any of the principle (or theory) that has been propounded in answers to his posts; he just doesn't think that it is practical in today's marketplace.

Again, this is not what I said. For the personal use of reasonably technically competent users, the SpamCop BL could be a useful option. Additionally, at institutions where a higher priority is placed on saving network resources and blocking spam, then it could certainly be used, with appropriate warnings to the user base.

Again, I'm not saying anything controversial. The SpamCop page itself says that the list is very aggressive and not necessarily suitable for production use. If you take issue with that, then feel free to take it up with the SpamCop implementors.

I think what's being missed here, is that you can achieve results of an at least equivalent and probably superior level as those of the SpamCop BL without being as destructive. No one list can do this for you, but selecting a few of most reliable lists that use different criteria is extremely effective.

I'm only trying to get at the last little bit of spam that's trickling through, while dealing with certain restrictions.

I don't think it's a great conspiracy that global whitelist doesn't exist (I'm not even convinced that such a thing doesn't exist), I just think it's too bad that I can't use the SpamCop BL, because I know it would be helpful.

If such a list existed, more people could use the list, more people might participate in the process, and everyone would win. I'm not trying to attack anyone here, and I really don't get the whole passive aggressive thing going on here.

All I really wanted to do was ask about the whitelist, but since that thread wasn't allowed, we got a little sidetracked.

I apologize for being disruptive.

Carry on.

Link to comment
Share on other sites

<snip>

F. Jones does not think that he can explain to his users who are members of yahoo groups why they cannot receive their groups any more.

I could certainly explain this, and they would understand, and then I would be told to undo what I'd done, because a higher priority has been placed on users receiving their mail than crusades, regardless of their value.

<snip>

...That's fine, provided the users are paying the full costs of the resources necessary to accept, forward, store (including backup media and resources necessary for the backup), deliver and otherwise manage all the useless stuff.

<snip> I think what's being missed here, is that you can achieve results of an at least equivalent and probably superior level as those of the SpamCop BL without being as destructive. No one list can do this for you, but selecting a few of most reliable lists that use different criteria is extremely effective.

I'm only trying to get at the last little bit of spam that's trickling through, while dealing with certain restrictions.

<snip>

...But aren't you failing to consider that you don't have to reject incoming e-mail based on the SpamCop BL ... you can use a more conservative BL to reject and use the SpamCop BL for filtering to direct suspected spam to an alternate "inbox," analogous to Yahoo!Mail's "Bulk Mail" folder?

Link to comment
Share on other sites

If F. Jones is still reading this, perhaps he will say whether I have summed it up correctly. Of course, he wouldn't phrase it quite the way I have. He would say that his approach is like politics 'the art of the possible' (based on ideals, but tailored to fit what can actually be accomplished in the real world).

Perhaps you didn't get this far. The 'real world' being people who want to accomplish something and are willing to deal with whitelisting spam sources and the resultant spam rather than put their energies into stopping spam (which is highly idealistic).

I apologize for being disruptive.

Carry on.

IMHO, you were not disruptive at all. The points that you are making (as you say), are held by a great many admins. And it is good for 'activists' to know that there are other valid opinions on spam control. The 'art of the possible' is what makes the world go round. However, the activist has a place also. And sometimes, debate between the two produces tangents that make the 'art' closer to the 'ideal' and all but the most rigid are happier.

QUOTE 

But because this is the spamcop forum and therefore a "hive of activism" no one has such a whitelist for him to use (besides not even comprehending why he would want such a list).

I'm not the one in this thread with a comprehension issue.

I think if you re-read this, you will see that's what I said.

I understand exactly what you want. You won't find it here because no one will admit that they would allow spam sources to be whitelisted (your term 'hive of activism'). And users like me want more control over what our IT department does (not that I actually have any or am likely to) so we wouldn't recommend a whitelist if we had access to one which we don't (tinw) because IT departments don't trust us.

But I am an eternal optimist and I think the tipping point will come.

Miss Betsy

Link to comment
Share on other sites

The last time a little while ago, after there were reports of Yahoo group servers being listed by the bl.spamcop.net, someone posted that Yahoo had modified the headers to indicate the true I.P. address that the e-mail originated from.

If you look at the news.admin.net-abuse.sightings for the yahoo servers for the older spams, you will find that the headers to indicate the true source.

About the time that the latest threads showed up, the spam in the .sightings newsgroup from yahoo shows that while they were putting the original I.P. in the headers, they were no longer putting in as a standard header, instead as a non-standard header line.

As there is no way to trust or verify who put a non-standard header, the parser is going to indicate the Yahoo server as the source.

Has anyone affected by this managed to contact Yahoo to find out why they stopped putting the standard headers on e-mails coming out from them?

-John

Personal Opinion Only

Link to comment
Share on other sites

...That's fine, provided the users are paying the full costs of the resources necessary to accept, forward, store (including backup media and resources necessary for the backup), deliver and otherwise manage all the useless stuff.

In this specific instance, those costs are minimal, relative to the potential loss of "real" mail. Also, most spam is being blocked in other ways anyway.

...But aren't you failing to consider that you don't have to reject incoming e-mail based on the SpamCop BL ...

No, I've mentioned that about half a dozen times in this thread.

If and when we implement a more complex solution, I'll surely take advantage of what SpamCop has to offer.

Has anyone affected by this managed to contact Yahoo to find out why they stopped putting the standard headers on e-mails coming out from them?

That's an excellent point. I'll fire off an e-mail and report back if anything interesting comes of it. It still doesn't solve the root of the "problem", but it would make things Better, which is always Good.

Link to comment
Share on other sites

Not all mail servers can be configured to use a DNSbl in a scoring system, and only have a choice of a reject or not based on a listing. That appears to be the case with the mail server F. Jones is using.

For those mail servers, using an aggressive DNSbl like spamcop.net will cause the rejection of real e-mail occassionally.

It does look like currently the spammers are finding new zombies faster than the conservative DNSbls are listing them.

So for the best coveraged of rejecting spam with out rejecting real e-mail, a two stage approach is needed, if your mail server can be configured to use it.

At the start of the SMTP transaction, use the conservate DNSbls to reject on I.P. addresses.

For the e-mail that passes that test, if the e-mail is not from an I.P. address listed in the more aggresive DNSbls and has the correct rDNS, pass it with only a possible virus check. A content fitler on these e-mails is more likely to throw a false positive than it is to detect a spam.

If a real incoming e-mail is coming from an address incorrectly listed in a DHCP pool, it will also likely have a bad rDNS, so that test combination can not be used absolutely.

If an I.P. address is listed in bl.spamcop.net and has either a bad rDNS or is listed in a DHCP pool, (or possibly SPEWS level 2 depending on your needs) lthen the probability of spam is high enough that it would be safe to reject.

For items only have enough characteristics to be suspicious, having a content filter that resolves the URLs to I.P. addresses and check those addresses against the sbl-xbl.spamhaus.org will probably safely reject the spam that the conservative lists.

If the URL does not resolve it is probably safe to reject an otherwise suspicious e-mail as spam. It is either a typo which the human sender will correct, or a spam site that has been kicked.

AOL.COM is reportedly not accepting e-mails with numeric I.P. URLs in them, and since they started that, I have not seen any spamers use them. And spammers are burning through domain names bought in bulk to guard against text based matching like those schemes that go by looking up reported spammed domains.

-John

Personal Opinion Only

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...