QUOTE(elind @ Jan 23 2005, 01:26 AM)
I have been a spamcop subscriber for several years now, and dutifully report every single spam received, and I read this and that article about spam, but I still don't understand the whole issue, or why we still have a problem.
For many users it is not that much of a problem. If you have a competent network administrator or postmaster, over 80% and up to 95% of the incoming spam can be rejected with out the risk of rejecting real e-mail, and that is before the spam ever leaves the sending machine.
This is by using the conservative DNSbls. Note that the spamcop.net DNSbl is not a conservative DNSbl.
This is not only the most effective way of blocking most of the spam, it is also the cheapest thing for a network operator to use. And in the rare case that a real e-mail is rejected, the sender gets notified by their ISP. While occasionally there have been errors with the conservative DNSbls, they are very rare.
If you then apply the state of the art in content filtering to the mail that makes it through the conservative DNSbls, almost all of the spam can be eliminated with out risk of a real e-mail being rejected.
The problem is that only SpamAssasin 3.0 is known to have those features, and not all mail servers can use SpamAssasin.
Just about all the mail servers can use the conservative DNSbls though to lower their oprating costs by reducing the incoming spam.
Also, even though the most accurate spam detection algorithm in SpamAssasin 3.0 has been known to the public internet for well over a year, it appears that none of the commercial spam filter vendors have adopted it.
Instead they seem to concentrate on spam filtering algorithms that have proven to be easily bypassed by spammers several years ago.
It is basically because the average ISP user does not understand this, they can end up paying more for bad service.
The other argument in favor of doing incompetent spam filtering is that filtering is censorship. The people making such statements are never the ones willing to pay the entire cash costs of what they are asking for. Instead they want it spread over thousands of users.
It appears that all of the mail servers that I get e-mail on now use at least some conservative DNSbls,
so the amount of spam I have to report is low, and mainly new zombie computers that have not yet made it into the sbl-xbl.spamhaus.org.
If my mail server operators systems could analyze the content of suspected spam before the SMTP transaction was over, and check the I.P. addresses of the URLs in them against the same I.P. addresses that they refuse e-mail from, then I would have almost no spam to report through spamcop.net.
What is know to work to remove the majority of spam has been known for a while, and what has been known to reliably remove the majority of the spam that gets though the blocking lists has also been known for over year.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand why 99% of all reputable ISPs can't have verified accounts that are allowed to send to mail lists, and stop all others that send anything looking like a mass mailing before it gets out the door.
That authentication is only useful for e-mail sent through the ISP's mail servers.
Most spam is sent directly from computers that have been infected with a remote control program and does not go through the ISP's mail servers.
So your next question would logically be why do not ISP's block mail from coming from those I.P. addresses?
First it would require the ISP's to keep track of what I.P. addresses are running servers and which are not. Of course many broadband ISP's prohibit all services on their home user I.P. addresses so that should not be an issue.
So the typical answer is that it would require all the mail servers that allow access to them from outside of their local network, which is a typical company mail server, to properly secure their mail servers for that type of access. And it appears that many companies still use insecure methods to have their remote users access their mail server.
Spammers look for those insecure servers and probe them with common username/password combinations. They seem to get into a large number of them that way. It is one of the most common ways for a real mail server to get listed on spamcop.net.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand why the biggest offenders, Chinese, Korean, Brazilian etc., are not simply blocked by all the others who want to be civilized (anyone who says the Chinese can't stop it overnight are ....).
As stated by another poster, some networks do that, and only white list mail servers in those countries by request of a one of their customers. Of one mail server operator I know that blocks by country, the rejection message text that accompanies the SMTP reject code explains how to request a whitelisting.
The last report I saw from that postmaster several years of operation with an international population of users, no one internal or external has requested an exception be made in a country specific block.
And there are many people who do not understand network management that would consider such blocks censorship.
So many ISP's do block e-mail from those countries, but instead of blocking it outright, they block the network segment that they received spam from. Ususally none of their customers ever notice.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand why spam reporting seems to have no effect. I do it because I "have faith", I suppose, but it has made no difference to my volume, unless constancy is considered a good thing.
If your mail server operator is not using at least the conservative DNSbls, and has only a content style filter for spam, then your reporting probably will not affect the amount of spam that you get.
Spam reporting does have an effect. At least one ISP who understands that every second a zombie computer on their network is present is costing them operating cash has stated that they have set up automated processing to handle spamcop.net reports to verify the report and isolate the infected machine. See the costs of spam pinned topic.
And from the last report of one of my postmasters, spamcop.net is only catching 3% of the spam delivery attempts, because it is only applied after the conservative blocking lists.
Based on reports on an internal user forum for by broadband ISP, anytime that any measurable quantity of spam is relayed through the ISP's mail servers instead of zombies, at least two major ISP's put those I.P. addresses on local blocking lists until they are convinced to remove them.
So a smart ISP realizes that a spam report from anywhere is something that needs immediate investigation.
My broadband ISP has stated on some forums that they are now receiving near realtime updates for several major blocking lists and looking for their I.P. addresses, so that they can act on spam/virus problems before they get a spam report.
And I do not just report spam through spamcop.
Most spam is now sent through open proxies, so I submit them to the MAPS-OPS and BOPM for processing. You have to get permission from the BOPM folks to submit to them, but MAPS-OPS just wants you to confirm that you will follow their rules for the first submission. The BOPM and MAP-OPS will accept reports in the same format.
To get permission to submit to the BOPM, you must read their FAQ and follow their instructions to the letter to show that you can understand basic instructions, and have a clue as to what you are doing.
The BOPM is part of the xbl.spamhaus.org, and these are considered conservative blocking lists and are used by far more mail servers than the spamcop.net blocking list because of that.
The spamcop.net parser also gives me the rDNS of the spam source, and if it is not an open proxy, the spamcop.net parser lets me know if it is in the SORBS dynamic list. And if the spam makes it through on one of my e-mail addresses, then I know that the source is not in the NJABL dynablock list.
So then if the rDNS has "pool", "dhcp","dyna","ppp", or "dial", in the name, this indicates that the spam came from a dynamic pool that is not known to one of the lists. When I am in a hurry, I only submit it to the dynamic list that mail server it went through was using. When I have time, I check NJABL, SORBS and MAPS-DUL and submit it to the ones that it is missing from.
MAPS-DUL requires a spam sample for them to consider an I.P. address. SORBS wants the rDNS to indicate that the I.P. address is clearly dynamic, and NJABL has not yet acknowledged any of my submissions, but I do not recall seeing any repeat spam from an I.P. block that I have submitted.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand why the spammers keep sending to addresses that end in "spamcop.net", or why they don't remove reporters from their lists.
They also do not remove "abuse" or "postmaster" from their lists, or even better "blockme" and "listme" which are common spamtrap e-mail addresses for some of the more aggressive DNSbls.
It appears from several investigation that the money is not in spam or the reponses to spam, but in selling spamware to victims that think they are going to get rich spamming.
Typically the victim spends their last $150 to over $1000 for a spammning kit, and a promise of payment on commisions. Then they spam like crazy until they either lose their ISP connection or finaly realize that they are never going to make back more than 10% of the money that they spent.
And every time some newspaper or TV show profiles the spammers that claim to be making money (with out verifying any of the claims), more victims line up to by spamming kits.
So basically much of the spam is being sent by people who have paid a lot of money to put a program on their computer that they have no idea of what it will do, and no way to determine if they will ever get paid. And even if they can find the con-artist that sold them the useless kit, in order to collect damages, they would have to admit that they bought it to make money by breaking the law.
And I have made many posts with serveral imaginary top level domains. Some of them showed up in the CC: list of spam that made it through to me as other intended recipiants because the first part of the e-mail name was the same. I have not seen any of them show up since my broadand ISP added DNSbls to their spam filtering which removed over 90% of the spam that their expensive content filter was not able to detect.
So the spamware is not even smart enough to remove top level domains that do not exist.
Other postmortems of captured spam databases show that the spammers harvest.anything.with.an[at]inside.ofit and that sample there will eventually show up in a spam database, and so will aaa.proof.of[at]us.canspam.violation eventually, hopefully alphabetized in a file entered into court evidence.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand why they suddenly started using random letter names on their forged emails instead of fake names.
Suddently?
You must have just been picked up on by a new group of spammers. Spammers have been using the random letter names for years.
The spammer is trying to avoid content filters, and this technique has been known to get through one of the most popular ant-spam defense that mail programs and spam filter vendors provide.
Now it has been well know for at least the past 8 years by anyone with a clue about filtering spam that filtering by alleged incoming e-mail address does not work, but it is still the most offered anti-spam solution. Either the companies offering such options are clueless, or they are just selling placebos to make it look lke they really care about their customers, even though they know it does not work.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand why they don't totally make up the forged email address, instead of using what mostly seems like real domain names (except for the stupid sender name).
Some poorly implemented spam filters operate on the forged sender name, so spammers will usually chose an ISP name that they think will usually be whitelisted.
Some mail servers will now probe the sending domain to see if the sending e-mail address exists prior to accepting the e-mail, if they have not seen e-mail from that user before.
Again, it is all something to bypass a spam filtering algorithm that should have been totally discarded almost a decade ago, but is still one of the most popular one to sell for money.
For the e-mail providers that I get most of my e-mail from, I can not whitelist by domain name. I have to request whitelisting by I.P. address, which is something that the spammers can not forge. And then I may have to explain why the sending I.P address is likely to be in a ranged blocked by that provider.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand if there is any point in reporting anymore, since it's been a long time that I saw a reply saying that such and such account had been closed down.
Use your favorite search engine for "Bedbug letter".
Spamcop.net by default suppresses automatic responses from the ISPs. Only a few ISP's actually right a personal reply when they kill an account.
The ones from Outblaze are the most interesting to read, but because of the anti-spam attitude of them, it is rare to get one. When Suresh fixes a problem it is seems to stay fixed.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
I don't understand how companies like Sprint can totally ignore the spam traffic that the Chinese pay them for, and still pretend to be be part of the civilized internet.
Sprint appears to be just one of many backbones that the Chinese have to connect in from. It appears to be one that is accepting reports. What they do with those reports, I do not know.
Steve Linford of spamhaus.org reports in news.admin.net-abuse.email that the situation in Chiina is improving as far as spam even though it might not look that way. And according to Chinese government press releases, just providing hosting for some of the types of web sites that I see advertised in spam can result in life imprisonment. But that asside, I have seen no change in the amount of spamvertised web sites that are in China.
QUOTE(elind @ Jan 23 2005, 01:26 AM)
Has anyone published a thorough, not excessively technical, document on these issues and the ones I haven't listed?
Are you volunteering to be a FAQ editor moderator
It also looks like it could be useful to have a topic that lists the various spam filtering methods in use, and discusses their strenghts and weaknesses. There have been various discussions about them on these forum, but not distilled down, especially for non-techies.
It could be used as a guide for those purchasing spam filtering software, so they could make an informed evaluation.
-John
Personal Opinion Only