Sign in to follow this  
Followers 0
macquigg

Why not whitelists

65 posts in this topic

IP blacklists aren't working. There are just too many IP addresses, and they are far too easy for spammers to acquire. Reputable domain names are harder to acquire, and far less numerous than IP addresses. Now that it is possible to connect a temporary IP to an authenticated domain name, it seems the next step is to switch to lists based on domain names. Is there any talk of SpamCop doing this? After years of discouragement, I now believe that the end of spam is in sight.

The system I have in mind would include a blacklist, a whitelist, and a greylist for new domains with no track record, but who promise never to spam. The vast majority of domains would just be "unknown", and that would be a forth category not on any list.

For this to work, I think you would need several things:

1) An easy transition from the current IP blacklists. ISPs using the new lists should have very little effort in reconfiguring their servers.

2) A good starting point for a whitelist, maybe all the largest domains on senderbase.org who have not been blacklisted.

3) Quick and easy feedback from users. At first there will be a large number of emails in the "grey" category. Listing a domain should be one click. That domain should go on the user's white/black list, and be automatically forwarded upstream for possible inclusion on the organization's lists, or on the world-wide lists maintained by SpamCop.

The same real-time technology now used for IP blacklists could be used to update the domain lists, only it would be much easier to manage, since the lists would be smaller and more stable.

These are just some initial thoughts as to how we might take advantage of the new authentication methods. If you aren't yet familiar with these methods, take a look at http://en.wikipedia.org/wiki/Email_Authentication

-- Dave :)

Share this post


Link to post
Share on other sites
There are just too many IP addresses, and they are far too easy for spammers to acquire.

It is even easier for a spammer to forge a domain from the whitelist and that does not cost him any money. I don't see how you are going to stop that, at least without a major overhaul to the email system (which means every machine that can send email in the whole world, a daunting task at best).

I've looked into several of the authentication schemes and do not think any one will catch on. Specifically, none of the ones I have looked into will allow me to use my email provider as the sender address but send email via my ISP. I have had too many problems with my addresses being added to mailing lists that NOBODY ever gets to see that address.

Also, I challenge the line in that article "why blacklists of spammer's IP Addresses aren't stopping spam". The problem as I see it is not enough people are using blacklists because "I might lose a message". A proberly configured blacklist will never "lose a message" because the sender will be informed of the problem. My blacklists (from the spamcop email system) are stopping more than 99% of my spam. If I get 2 spam in one day in my inbox, something strange is happening, yet it catches between 125-200 messages in the Held Mail folder each and every day. The combination of effective blacklists associated with a whitelisting system is the best solution for most people.

Share this post


Link to post
Share on other sites
IP blacklists aren't working.

Based on traffic in NANAE, other newsgroups, even here, I'd say that they do work. How they are implemented is the real issue.

There are just too many IP addresses,

Actually that number under IPv4 is finite, and even all of those aren't actaully available/used.

and they are far too easy for spammers to acquire.  Reputable domain names are harder to acquire, and far less numerous than IP addresses.

Now that's just a bit silly. Define "reputable domain names" please. Virtual hosts that have thousands of domain names pointed to the same server. Actually "obtaining" an IP is usually pretty expensive, as compared to the norm of a spammer "using" an IP connected to a compromised system ..which is not the same thing at all .... Some of the top spammers have been known to burn 20 to 50 domain names a day ...

  Now that it is possible to connect a temporary IP to an authenticated domain name,

You've actually got that totally backwards (ignoring a spoof mode, but that doesn't seem to fit your rant ..??)

it seems the next step is to switch to lists based on domain names.

Look for something along the lines of SURBL ....

After years of discouragement, I now believe that the end of spam is in sight.

Optimist?

Share this post


Link to post
Share on other sites

Also, I challenge the line in that article "why blacklists of spammer's IP Addresses aren't stopping spam".  The problem as I see it is not enough people are using blacklists because "I might lose a message".  A proberly configured blacklist will never "lose a message" because the sender will be informed of the problem.  My blacklists (from the spamcop email system) are stopping more than 99% of my spam.  If I get 2 spam in one day in my inbox, something strange is happening, yet it catches between 125-200 messages in the Held Mail folder each and every day.  The combination of effective blacklists associated with a whitelisting system is the best solution for most people.

24436[/snapback]

Yep the answer or solution to everyone's problem is to get the only email address you will ever need no email is ever lost by using SpamCops email system's blocking lists (mainly because it is thought out and set up correct)

I get about 1 spam a month in my inbox which is easily dragged to my reporting folder (using IMAP)

IMAP can also be used to easily whitelist email addresses by dragging them to reporting folder and then released by "Send and Whitelist" "button"

Share this post


Link to post
Share on other sites
It is even easier for a spammer to forge a domain from the whitelist and that does not cost him any money.  I don't see how you are going to stop that, at least without a major overhaul to the email system (which means every machine that can send email in the whole world, a daunting task at best).

Forging a domain name will not be possible unless the receiver fails to authenticate the name. The "major overhaul" is that legitimate senders will need to register their authorized sending IPs, and receivers will need to check the IPs on incoming packets against the purported domain name. We don't need to worry about every machine in the world. Those that don't register can be ignored. Those that register and spam will be quickly blacklisted, and they will find it ever more difficult to acquire a reputable name, or register a new name and con an organization like SpamCop into giving them a good initial rating. Those who ignore the rating system can drown in their own sewage, and it won't interfere at all with those who do use authentication. We will just "tune them out".

I know this brief explanation is not enough to convince you of the technical validity of what I'm saying, but if you read the article http://en.wikipedia.org/wiki/Email_Authentication I think you will understand.

I've looked into several of the authentication schemes and do not think any one will catch on.  Specifically, none of the ones I have looked into will allow me to use my email provider as the sender address but send email via my ISP.  I have had too many problems with my addresses being added to mailing lists that NOBODY ever gets to see that address.

I'm an electronic design engineer, not a computer expert, but I do have a good book on internet protocols (Stevens, TCP/IP Illustrated). I think the problem you are referring to is the fact that many email forwarders do not currently verify the IP addresses of incoming mail. The good ones do, however, and the others will have to change their procedure, or lose customers like me who won't tolerate them acting as anonymous relays for spam.

Also, I challenge the line in that article "why blacklists of spammer's IP Addresses aren't stopping spam".  The problem as I see it is not enough people are using blacklists because "I might lose a message".  A proberly configured blacklist will never "lose a message" because the sender will be informed of the problem.  My blacklists (from the spamcop email system) are stopping more than 99% of my spam.  If I get 2 spam in one day in my inbox, something strange is happening, yet it catches between 125-200 messages in the Held Mail folder each and every day.  The combination of effective blacklists associated with a whitelisting system is the best solution for most people.

I agree it's gong to take a combination of black, white, and even grey lists. The essential difference is that we replace rapidy changing IP addresses with much smaller and more stable lists of domain names. Then email recipients will gain the upper hand over spammers, and will be much more enthusiastic about reporting any spam that slips through. A big onslaught of spam from a reputable domain could get the entire domain blocked within minutes, and a rapid response from the domain owner.

On the issue of informing the sender that a message was blocked, I think that authentication of domain names will avoid the problems we have now with C/R systems. When you send a block message *only* to the domain that owns the IP address from which the spam was sent, the worst it can do is bother someone in the same domain as the spammer. They will complain to the domain owner, and the owner will have an incentive to shut down the spammer.

Share this post


Link to post
Share on other sites
I know this brief explanation is not enough to convince you of the technical validity of what I'm saying, but if you read the article http://en.wikipedia.org/wiki/Email_Authentication I think you will understand.

I have read the article and the supporting and dissenting documentation. Have you read the dissenting opinions, specifically this one http://homepages.tesco.net./~J.deBoynePoll...is-harmful.html which I believe to be the more readable? I see more downsides than upsides.

I think the problem you are referring to is the fact that many email forwarders do not currently verify the IP addresses of incoming mail. The good ones do, however, and the others will have to change their procedure, or lose customers like me who won't tolerate them acting as anonymous relays for spam.

No, SPF is trying to tell me where I want my return mail to go, which in my configuration is NEVER back to the ISP I sent it through as I do not use them for receiving email.

What I think is going to happen on the sending side, because of the problems people will have getting their messages through if it starts to be implemented on the receiving side, is most people will configure it allowing ANY host to send on their behalf, thereby making it virtually useless. Again, if you could get the whole world to switch immediately, it is possible it could work, but the former is not likely. And if immediate switch were possible, there are far better answers out there.

Share this post


Link to post
Share on other sites
I have read the article and the supporting and dissenting documentation.  Have you read the dissenting opinions, specifically this one http://homepages.tesco.net./~J.deBoynePoll...is-harmful.html which I believe to be the more readable?  I see more downsides than upsides.

Yes, I read that aricle, but I was not able to follow the substance of the arguments. It seems like a very emotional argument (lots of loaded words). Then again, I am not an expert. The one item that did seem plausible was the problem with forwarders.

The encyclopedia article tries to explain that in more fundamental terms. There are certain essential pieces of information that the forwarder must preserve in order to allow the receiver to authenticate the sender. As I understand it, mail forwarders do not now preserve the IP Address of the incoming mail. That to me is fundamentally "broken", as it will provide a opportunity for spammers to hide their identity. This needs to be fixed, regardless of which method is used for authentication.

No, SPF is trying to tell me where I want my return mail to go, which in my configuration is NEVER back to the ISP I sent it through as I do not use them for receiving email.

I think the way this *should* work is that anything rejected at any stage should go straight back the path it came, while return mail from the recipient should go to the From: address, or the Reply-To: address, if that is available. Those decisions are made by the recipients email program. The recipient should never be concerned about the forwarder.

Forwarders that are now "broken" will have to learn how to handle rejects, or lose customers like me. There are lots of forwarders that do it right.

There is a diagram showing how SPF handles forwading at http://spf.pobox.com/srspng.html This seems un-necessarily complex to me, but it doesn't matter, as long as the essential information is forwarded, the receiver can perform authentication on the sender if necessary.

Share this post


Link to post
Share on other sites

24448[/snapback]

A reason why DUL is never used is that all email Headers have a DUL IP in them meaning all email would fail unless whitelisted

A spammer only has to call his computer ""Whitelist IP" and regardless of where it was sent from because it had a "whitelist IP" mentioned in headers it would go to ones inbox (whitelists over ride all blocklists)

It may be certain POP servers could identify a IP in a specific manner but most just "Keyword search headers" first for whitelist (if none) then for blacklist

Edited by petzl

Share this post


Link to post
Share on other sites
A spammer only has to call his computer ""Whitelist IP" and regardless of where it was sent from because it had a "whitelist IP" mentioned in headers it would go to ones inbox (whitelists over ride all blocklists)

Domain lists work *only* if the domain name is authenticated. That is why they have not been possible until now.

It may be certain POP servers could identify a IP in a specific manner but most just "Keyword search headers" first for whitelist (if none) then for blacklist

Also, the receiver must use the *authenticated* name in matching his white/black lists, not some other name that the spammer might have forged.

Share this post


Link to post
Share on other sites
Then again, I am not an expert.

For someone claiming that you are not an expert, you seem to have all the answers ;)

Share this post


Link to post
Share on other sites

Totally from the other side of the fence, a recent posting over in the support forum for this application included the follwing (and , geeze, I couldn't help but think of your postings here <g>)

I just bought and registered another 15 Domain names .... the goal is to get good Serach Engine result placement ..... (he then lists all the domains in questions and is asking for input on how best to make sure that Google bot visits will "correctly" identify that all of these 15 new Domains all "point" to the "one 'good'" URL ...)

and again, you've still not defined "reasonable Domain names" ... and you still haven't caught the flip side of the coin in that you can "place" a web-site anywhere, it's just that it can only be found if one has a DNS server pointing it out .. and that's a whole nother Topic already in discussion elsewhere ....

Share this post


Link to post
Share on other sites
Totally from the other side of the fence, a recent posting over in the support forum for this application included the follwing (and , geeze, I couldn't help but think of your postings here <g>)

and again, you've still not defined "reasonable Domain names" ... and you still haven't caught the flip side of the coin in that you can "place" a web-site anywhere, it's just that it can only be found if one has a DNS server pointing it out .. and that's a whole nother Topic already in discussion elsewhere ....

24469[/snapback]

A "reputable domain name" will be whatever the manager of the domain-rating list decides it is. SpamCop might have one definition. The Council-of-Churches might have another. The US government might *legislate* a definition if the industry doesn't clean up this mess soon. I can just imagine the FCC deciding what is "obscene" and fining ISPs that don't use their list.

If I were putting together an initial list, I would take the domains with the largest flow of email, and rank them by % of spam. Then I would grade them, with an "A" rating being maybe less than 1%. I have no idea what the % is for low-spam domains like aol.com, but I'll bet it is less than 1%.

The initial list would require real-time updates, like SpamCop does now with IP blacklists. Spamcop should be in an excellent positition to put together a list like this, since they are getting a huge amount of raw data directly from their subscribers. I see also that senderbase.org has good stats on total flow, but they don't have anything on the % of spam.

I'm still not understanding your comment about the "flip side" of DNS. As I understand it, traditional DNS lists only the *receiving* IP address of a domain. The new SPF records list the authorized *sending* addresses. It is those sending addresses that are checked to see if a domain name is forged.

I don't claim to have all the answers. I do know a few things, but my study of this problem is recent, and there are gaps in my understanding. That is why I participate in forums like this, where I hope to find experts with a lot more experience, and some willingness to share their knowledge. I listen carefully, but I don't take anyone's word for something. A year ago I had a discussion in comp.security.misc, and the experts there informed me that it was utterly impossible to identify the sender of an email. Now that I know something about TCP/IP protocols, I can see for myself that isn't true. Too many "experts" just repeat what they have heard, not what they truly understand.

-- Dave

Share this post


Link to post
Share on other sites

I haven't followed this thread very closely, but identifying 'whitehats' is supposed to be too labor intensive to be practical, IIRC other discussions.

There is no possibility of government getting involved, IMHO, because of the international aspect as well as the manpower needed for enforcement. (not to mention lobbying groups that don't want the government involved).

IIRC, there are already a couple of outfits that are trying to create 'whitelists' - Bonded Senders, Habeus. I think that whitehats should get a lower score in the spamcop algorithym so they age off quickly after they have fixed the problem. And that is another problem - mistakes happen and spammers get by even whitehats.

The blacklists would also work like a whitelist if everyone used lists that blocked at the server instead of using content filters. Trouble is they don't. End users need to be educated that 'yes, you can find the sender of an email' (except through open proxies but if it is spam, there has to be a website or some kind of contact information so they can get their money) and that the *sending* end needs to be responsible and competent. If an end user's email is blocked, then they are responsible for finding a good way to send it. The recipient should never have to do anything except report spam to his ISP who immediately blocks that IP address for x hours.

Then there would be two networks - one that has irresponsible, greedy, or incompetent ISPs and one that has whitehats. And no one in the latter would ever accept email from the former.

My $.02 USD

Miss Betsy

Share this post


Link to post
Share on other sites
I haven't followed this thread very closely, but identifying 'whitehats' is supposed to be too labor intensive to be practical, IIRC other discussions.

Whitehats would earn their status by demonstrating a low ratio of spam to good mail over a period of time, say 30 days. This could be done without any more labor than is currently involved in spam reporting. That would earn them an "A" rating, and give them incentive to keep their spam under control.

"D" rated domains (unrepentent spammers) would be a blacklist, just like we have now, only with domain-names instead of IP addresses.

"C" rated domains (unknown) would be no labor at all. They simply aren't listed.

"B" rated domains (trusted, but not yet proven) would be the hardest. For those you might want to charge a small registration fee, basically to cover the cost of checking their corporate records, or in the case of individuals, a credit report and a check of criminal records. I don't think many individuals will need to operate public mail servers.

IIRC, there are already a couple of outfits that are trying to create 'whitelists'  - Bonded Senders, Habeus.  I think that whitehats should get a lower score in the spamcop algorithym so they age off quickly after they have fixed the problem.  And that is another problem - mistakes happen and spammers get by even whitehats.

Bonded Senders is a good example, but this is high-dollar, high-security, big corporation stuff. SpamCop could reach a much larger clientelle, where security is not so vital. If you let a few spammers through, it doesn't mean somebody has to forfeight a bond.

The blacklists would also work like a whitelist if everyone used lists that blocked at the server instead of using content filters.  Trouble is they don't.  End users need to be educated that 'yes, you can find the sender of an email' (except through open proxies but if it is spam, there has to be a website or some kind of contact information so they can get their money) and that the *sending* end needs to be responsible and competent.  If an end user's email is blocked, then they are responsible for finding a good way to send it.  The recipient should never have to do anything except report spam to his ISP who immediately blocks that IP address for x hours.

This is the current business model, and it isn't working. The problem is not technology, but social engineering. We need to engineer a system that has positive feedback at every point along the growth curve, encouraging ISPs who don't want to be bothered, to make the effort and clean up their domains.

We now have plenty of domains publishing their SPF records, but a lag in mail receivers checking those records and rejecting senders who don't authenticate.

What we need now is a good domain-rating list, one that will provide a demonstrable reduction in the number of emails needing further processing by a spam filter, and for those emails that do need further processing, a domain-rating to be factored into the spam score. When I convice my ISP to give it a try, I want the reaction to be "That's cool. Where do I get this list?" That still won't cause him to clean up his outgoing spam, but if other ISPs do the same, he will be getting some of their rejects, and then he will clean house.

Then there would be two networks - one that has irresponsible, greedy, or incompetent ISPs and one that has whitehats.  And no one in the latter would ever accept email from the former.

It's fascinating to think about the ramifications of this. What will happen to the "undisciplined" domains on the internet? Will they simply suffocate in their own excrement, or will they work out some "code of ethics" to keep the sewer snoids from ruining the party. Maybe when a piece of spam arrives, an ISP could respond "I have 15,322 recipients willing to receive your message, which I will deliver for $.02 each." Of course, that assumes the spammer authenticates, so maybe we will see a third network, totally unregulated, populated by folks who scribble on the walls in men's restrooms. Won't bother me a bit. The bandwidth of the internet is unlimited.

Share this post


Link to post
Share on other sites
The bandwidth of the internet is unlimited.

24487[/snapback]

Now there's a whopper if I ever saw one!

Share this post


Link to post
Share on other sites
Now there's a whopper if I ever saw one!

Technically, you are right, but did you get my point? Let me re-state more precisely:

Unlike the bandwidth of the broadcast spectrum, the "bandwidth" of the internet has no inherent limit. If spam continues its current exponential growth, in 8 years we will have 10 times the traffic on the Internet that we have now. The providers of bandwidth will be happy to add that capacity. The limit is not bandwidth, but how much people are willing to tolerate the "noise". By separating the legitimate domains from the spammers, we can enjoy a "clear channel", even when the "signal to noise ratio" on unfiltered receivers is 100 to 1.

My point is I have no problem with spammers using all the bandwidth they are willing to pay for. We are not fighting over limited bandwidth, but over their ability to jam my communications, forge my name, and interfere with my business.

Is anyone here willing to think out of the box that we are now in?

-- Dave :angry:

Share this post


Link to post
Share on other sites

I'm starting a fresh topic here, so we can talk about what might happen in the near future if I am right about issues of technical feasibility. Let's discuss those issues in the thread "Why not whitelists?" For this thread, let's assume that email authentication works, and is widely adopted by email receivers. Spammers can no longer forge the name of any domain that doesn't allow it.

How will email evolve? What problems will arise? Will the internet separate into isolated non-interacting "walled gardens"? Will the part outside the walls become so polluted as to be useless? Or will spam die out because there are so few recipients that spammers can't even get their 15 in a million to break even? I'm sure you can think of other interesting questions.

-- Dave

Share this post


Link to post
Share on other sites

I agree that social engineering is required instead of technology. I disagree that there needs to be additional technology to make spam a 'non-problem'

I don't understand the insistence on 'domain names' rather than IP addresses. There are some people who think that spam blocking would work better by blocking on the spamvertised site rather than IP address and IIRC, they have set up a blocklist.

The main problem is that the end user (who is paying for the email system) has not been educated that it is hir responsibility to choose a reliable ISP who does not get blocked often or for long. Nobody else has enough financial leverage or the numbers to create public relational nightmares for the backbones and registrars to really *do* something about spam.

Actually, my pet scheme to end spam is to make senders identify bulk email in the headers (already an RFC). All bulk email would be blocked unless whitelisted by end user (whitelists are already in place in many email systems) and that would be one additional step in the confirmation process. Any email that was received that seemed to be bulk would be reported and that IP address blocked for a specific period of time (using a nanas to confirm bulkness and like spamcop, assuming that transgressions will be fixed). The end result is that IP addresses that consistently ignored the requirement of bulk headers for bulk email would be blocked. If an end user wanted to get spam or use content filters because of not wanting to miss an email, he could turn the whitelisting off (but possibly have to pay an increased user fee because of the extra volume). Everybody else would get the bulk emails they want to get and would have no worries about missing an individual email since they would receive everything that wasn't bulk or it would be rejected at the server level so their correspondents would know that their ISP was incompetent or irresponsible (or it was one of those glitches that happens like being caught in traffic on the interstate).

No one likes it because it requires 'educating' ISPs all over again and puts the burden of controlling spammers even more squarely on the ISPs' shoulders by making them responsible for their customers knowing how to use the internet. IMHO, the technical reasons that it would be infeasible are like the reasons used for continuing to use 'misdirected bounces'

Miss Betsy

Share this post


Link to post
Share on other sites
I'm starting a fresh topic here,

24492[/snapback]

I'm not sure why the post above was moved to this topic, but I assume the administrator wants to keep it all in one place. That's OK with me if we can move the discussion in a positive direction and talk about what might be possible if I am right about the new authentication methods. I could spend my time reading IETF drafts, but I learn things more quickly, and more enjoyably in discussions like this.

We just need to avoid insults, quotes out of context to score a debating point, etc.

-- Dave

Share this post


Link to post
Share on other sites

If you are going to talk to sys admins about technical subjects, then you had better get used to insults. They aren't really insults anyway because sys admins use what some people consider rude, insulting language when they discuss issues and don't really mean anything personal by it.

Also, they tend to be highly opinionated and not easy to convince and definitely 'anal' in their attitudes toward precise wording and precise description of procedure (and that comes direct from the horse's mouth which is a little bit of an oxymoron). They also tend to sound flippant, but again, one needs to ignore that and concentrate on the content.

AFA debating points, a debate does tend to be a learning experience because one has to be able to defend one's position. However, debates do tend not to be 'fair' (i.e. taking things out of context), but that's when 'you' tell them that they need to go to reading comprehension school (or ignore it, since anyone who knows anything can easily see that it was taken out of context).

I, too, learn better from discussion than by just reading. You might concentrate on the parts that you don't understand why there is an objection. That's how I learned about the differences between an emailed bounce and rejection at the server level. I also learned from another discussion that sys admins consider 'filtering' to be done both before and after the SMTP part (and now I have forgotten the tech term). Blocklists are just another form of filter. That may not be pertinent to this discussion. Another aspect that you might take into account is that most sys admins have the attitude, 'my server, my rules' I don't know what it is that they object to in your proposal, but there may be some aspect that what it is you are proposing is unnecessary because those who know, don't need it and those who don't, won't use it.

Miss Betsy

Share this post


Link to post
Share on other sites
I agree that social engineering is required instead of technology.  I disagree that there needs to be additional technology to make spam a 'non-problem'

I would not rule out small changes in technology, however. If someone comes up with a better filtering algorithm, and it fits right in with our existing spam filters, no problem. What I like about the "new technology" of authentication is that it is so simple, and so little departure from existing technology. The essential information is kept in DNS records. Using that information requires only a query like is already being done by any public mail server. Most mail receivers won't have to do anything but enable an option in their mail programs, and they will have plenty of incentive to do that. Mail senders have a bit more of a burden, and a lower incentive, and that is where the social engineering will be necessary.

I don't understand the insistence on 'domain names' rather than IP addresses.  There are some people who think that spam blocking would work better by blocking on the spamvertised site rather than IP address and IIRC, they have set up a blocklist.

IP addresses are cheap, and will be even cheaper when the address space expands from 4 billion to some astronomical number. Reputable domain names are not cheap, and will be impossible to forge.

The main problem is that the end user (who is paying for the email system) has not been educated that it is hir responsibility to choose a reliable ISP who does not get blocked often or for long.  Nobody else has enough financial leverage or the numbers to create public relational nightmares for the backbones and registrars to really *do* something about spam.

I just don't see how we are going to educate all these users to be the drivers of change. Even users like myself who understand the problem have very little influence on ISPs. My ISP really doesn't care. They run a spam filter, because they can put that in ads for new subscribers, but other than that, they would rather not even think about the problem.

Backbones and registrars have even less incentive to get involved. Backbones deal in data packets and routing tables. They can't tell if a packet is even part of an email, let alone if it is spam. Registrars have a responsibility to ensure the integrity of the records for each domain, but will shun assuming any liability for rating those domains. The best source of a good domain-rating list will be private companies like SpamCop, whose only responsibility is to their subscribers.

Actually, my pet scheme to end spam is to make senders identify bulk email in the headers (already an RFC).  All bulk email would be blocked unless whitelisted by end user (whitelists are already in place in many email systems) and that would be one additional step in the confirmation process.  Any email that was received that seemed to be bulk would be reported and that IP address blocked for a specific period of time (using  a nanas to confirm bulkness and like spamcop, assuming that transgressions will be fixed).  The end result is that IP addresses that consistently ignored the requirement of bulk headers for bulk email would be blocked.  If an end user wanted to get spam or use content filters because of not wanting to miss an email, he could turn the whitelisting off (but possibly have to pay an increased user fee because of the extra volume).  Everybody else would get the bulk emails they want to get and would have no worries about missing an individual email since they would receive everything that wasn't  bulk or it would be rejected at the server level so their correspondents would know that their ISP was incompetent or irresponsible (or it was one of those glitches that happens like being caught in traffic on the interstate).

I'm not seeing what's new here. The labeling of spam is already *mandated* in CAN spam, and ignored by spammers. Blacklisting their IPs is an endless game of "whack-a-mole". What we need is an order-of-magnitude increase in the effectiveness of spam blocking. That will get the attention of the spam-tolerant ISPs.

No one likes it because it requires 'educating' ISPs all over again and puts the burden of controlling spammers even more squarely on the ISPs' shoulders by making them responsible for their customers knowing how to use the internet.  IMHO, the technical reasons that it would be infeasible are like the reasons used for continuing to use 'misdirected bounces'

With authentication there will be no more misdirected bounces. The bounce goes to the postmaster at the domain which actually sent the spam, and optionally to the alleged "from" individual in that domain. This will be at the discretion of the spamee. I will be sending personal messages, not easily ignored as bounces. If the alleged "from" individual is not the spammer, he can complain to his own postmaster. We know it came from his domain.

-- Dave

Share this post


Link to post
Share on other sites
IP addresses are cheap, and will be even cheaper when the address space expands from 4 billion to some astronomical number. Reputable domain names are not cheap, and will be impossible to forge.

The problem that I see is that lots of domain names share a single server. I don't know much about webhosting, but I think that there would be lots of people who would be very upset if, in order to own a domain, one had to run a mail server.

I just don't see how we are going to educate all these users to be the drivers of change.

The logical people are the ISPs. A good trade association would be a great benefit. Obviously the ISPs are not interested.

I'm not seeing what's new here. The labeling of spam is already *mandated* in CAN spam,

But that is 'content' not in the headers which, I would think, would be readable at the server level. It is also not just spam that would be labled, but *all* bulk email.

The other new thing is that it takes spam away from being a debatable issue (about advertising, etc., freedom to send and receive, etc.) and puts it as 'responsible and competent' or 'irresponsible or incompetent' The newness is that *all* bulk email would be blocked unless specifically whitelisted. That puts the burden on the mailing list manager to do it right and on the ISP to see that it is done right. The only email that would be blocked otherwise are those IP addresses where the admin does not make sure that bulk email is properly labeled.

So the spammers could send as much spam as they wanted as long as they labeled it bulk. The 1% who buys from them could receive it by not using the default block. And if legitimate email was blocked, it was because the ISP allowed 'irregular' email to be sent - not anything to do with content, but because the bulk email header line was not used. That puts the 'innocent' sender as being accused of using an incompetent ISP, not of sending spam. For those end users who use an ISP who uses the blocklist, if the ISP has not done a good job of educating them to report spam for the blocklist, then when they complain about the blocklist, there is another opportunity for the ISP to educate so that the recipient then will complain to the sender that they need to use a more reliable ISP.

It would probably drive up the price of sending bulk email because of the precautions ISPs would have to use to ensure bulk emailers use the header. It might also drive up the price of receiving email if one doesn't use the whitelist. Both would discourage spam since the first would make sending bulk email more expensive and the second would reduce the number of customers.

It would also encourage ISPs to act against trojanned machines because their other users would object, not because they were allowing spam to be sent, but because they were incompetent in preventing blocking.

However, individual emails would not be blocked unless they came from an ISP who did not put safeguards into place that prevented bulk emailers from sending mailings without the email header. As long as a sender used a reliable ISP, their emails would always go through.

Registrars have a responsibility to ensure the integrity of the records for each domain, but will shun assuming any liability for rating those domains.

There is a great danger in rating domains because of 'content' However, even with your system, it would be much better if the domain was rated as being 'competent' or 'incompetent' in using internet conventions.

Miss Betsy

Share this post


Link to post
Share on other sites
Is anyone here willing to think out of the box that we are now in?

Most folks here deal with the reality of the hardware, software, and folks existing and in use today. The magic box, the magic code, all people involved at every level may know all that needs to be known tomorrow ... however, there is no way that the "net" would change tomorrow afternoon or even the day after ....

Unlike the bandwidth of the broadcast spectrum, the "bandwidth" of the internet has no inherent limit.

You might want to explain various sites that monitor 'net' traffic if there is "no limit" .... a few offered;

http://internetpulse.net/

http://www.internettrafficreport.com/main.htm

http://www.noc.ucla.edu/weather.html

http://weather.uci.edu/

http://www.bluehill.com/weather/

The providers of bandwidth will be happy to add that capacity.

You might want to do some reseach on the term "dark fiber" ....

I'm not sure why the post above was moved to this topic, but I assume the administrator wants to keep it all in one place.  That's OK with me if we can move the discussion in a positive direction and talk about what might be possible if I am right about the new authentication methods.  I could spend my time  reading IETF drafts, but I learn things more quickly, and more enjoyably in discussions like this.

We just need to avoid insults, quotes out of context to score a debating point, etc.

This Moderator sees a lot of your points as so far off the mark at this point, he couldn't follow that yet another Topic needed to be started ... as so many points in this one still seem to be unresolved. As far as insults, debate points, and such, I'd suggest that you started that with the above remark/question.

Most mail receivers won't have to do anything but enable an option in their mail programs,

One could say the same thing right now about various security and configuration issues with a lot of the software out there today, but ..... As above, that not all software in use is current, not all installed by folks that can read/inderstand the documention, on and on .... you've made yet another general statement that really can't be seen as especially true ...

IP addresses are cheap,

You keep repeating this as your mantra ... Please give me the "cheap" source for something like 100 contiguous IP addresses.

Reputable domain names are not cheap, and will be impossible to forge.

And once again "reputable" is in play ...??? For that matter, when's the last time you actually tried to register a "simple" Domain name? Some Hosting services offer "free" Domain registration as part of the Hosting package, other ISPs offer $3 to $7 as a yearly fee dor a Domain name/registration ... I am having a hard time going with your repeated "expensive" description for creating/registering Domain names. Your "impossible" word here also begs an argument.

I just don't see how we are going to educate all these users to be the drivers of change.  Even users like myself who understand the problem have very little influence on ISPs.  My ISP really doesn't care.  They run a spam filter, because they can put that in ads for new subscribers, but other than that, they would rather not even think about the problem.  .... I'm  not seeing what's new here.  The labeling of spam is already *mandated* in CAN spam, and ignored by spammers.  Blacklisting their IPs is an endless game of "whack-a-mole".  What we need is an order-of-magnitude increase in the effectiveness of spam blocking.  That will get the attention of the spam-tolerant ISPs.

Perhpas you need to spend some time in NANAE, reading posts from ISPs that want to be removed from BLs ... Perhpas you need to read some of the posts here in the BL Forum section from folks asking to be removed from the SpamCopDNSbl ...??? I'm having a bit of a hard time in following your logic here ...

With authentication there will be no more misdirected bounces.  The bounce goes to the postmaster at the domain which actually sent the spam, and optionally to the alleged "from" individual in that domain.  This will be at the discretion of the spamee.  I will be sending personal messages, not easily ignored as bounces.  If the alleged "from" individual is not the spammer, he can complain to his own postmaster.  We know it came from his domain.

The words "if" .. " someday" .. "wouldn't it be nice" .. come to mind. But, so does "reality" ....

Share this post


Link to post
Share on other sites
The problem that I see is that lots of domain names share a single server.  I don't know much about webhosting, but I think that there would be lots of people who would be very upset if, in order to own a domain, one had to run a mail server.

You wouldn't need a mail server to own a domain, just a single IP address, most likely one allocated from your ISP's netblock. As they are done now, most of these dinky little domains would be just websites (not even an internal mail server), hosted on one of the ISP's machines, with 24/7 maintenance, etc. The burden of having to buy a separate machine might come when the owner of dinky.com decides, for some reason, he must have his own public mail server. I say "might", because I think it is actually possible, using network address translation in a router, to have multiple mail servers with different public IPs on one machine. In any case, the cost of a machine should not be an issue to anyone serious about operating a public mail server.

The other new thing is that it takes spam away from being a debatable issue (about advertising, etc., freedom to send and receive, etc.) and puts it as 'responsible and competent' or 'irresponsible or incompetent'  The newness is that *all* bulk email would be blocked unless specifically whitelisted.  That puts the burden on the mailing list manager to do it right and on the ISP to see that it is done right.  The only email that would be blocked otherwise are those IP addresses where the admin does not make sure that bulk email is properly labeled.

I like this labeling idea, and getting away from having some fixed definition of what is spam. That is what CAN-spam was supposed to do, but it failed. Spammers make more money by ignoring the rules, and there is essentially no enforcement.

So the spammers could send as much spam as they wanted as long as they labeled it bulk.  The 1% who buys from them could receive it by not using the default block.  And if legitimate email was blocked, it was because the ISP allowed 'irregular' email to be sent - not anything to do with content, but because the bulk email header line was not used.  That puts the 'innocent' sender as being accused of using an incompetent ISP, not of sending spam.  For those end users who use an ISP who uses the blocklist, if the ISP has not done a good job of educating them to report spam for the blocklist, then when they complain about the blocklist, there is another opportunity for the ISP to educate so that the recipient then will complain to the sender that they need to use a more reliable ISP.

Users now feel helpless. Reporting spam is futile. When I report what looks like a pretty serious attempt to defraud clients of smithbarney.com, Smith Barney does nothing!! It's not that they don't care that their name and logo are being forged. They just don't think anything can be done about it.

I believe this will change when email forgery stops, and domain owners can be held responsible for what comes out of their domains. Then comcast.net will decide if it wants to be, in the eyes of email recipients, more like aol.com or like china.net. If every recipient sees the true domain on every arriving email, name reputation will become a very important business asset, and those who try to rip off a name will face serious civil and criminal penalties.

It would probably drive up the price of sending bulk email because of the precautions ISPs would have to use to ensure bulk emailers use the header.  It might also drive up the price of receiving email if one doesn't use the whitelist.  Both would discourage spam since the first would make sending bulk email more expensive and the second would reduce the number of customers.

It would also encourage ISPs to act against trojanned machines because their other users would object, not because they were allowing spam to be sent, but because they were incompetent in preventing blocking.

However, individual emails would not be blocked unless they came from an ISP who did not put safeguards into place that prevented bulk emailers from sending mailings without the email header.  As long as a sender used a reliable ISP, their emails would always go through.

I think what will probably evolve is a system where every ISP has at least one A-rated domain name which sends no bulk anything, and a few other domains that they are willing to take a small risk of getting a lowered rating. The value of a high rating, and the difficulty of earning one, will ensure that ISPs review carefully any requests to send bulk mail that looks like spam.

There is a great danger in rating domains because of 'content'  However, even with your system, it would be much better if the domain was rated as being 'competent' or  'incompetent' in using internet conventions.

Only the recipients can rate content. A domain-rating service would simply go by the stats. If your domain has shown 10% spam over the last 30 days, that results in a certain rating change. I can't imagine much argument over this, unless the spam reports are all coming from one source.

Share this post


Link to post
Share on other sites
You might want to explain various sites that monitor 'net' traffic if there is "no limit" .... a few offered;

It looks like I miscommunicated. I apologize for assuming the responses were merely "scoring points". I'm seeing a fundamental difference between the bandwidth of the radio spectrum, and the "bandwidth" of the internet, a difference that has profound implications for the question of what to do about spam. Maybe we should just leave it at that.

You might want to do some reseach on the term "dark fiber" ....

Dark fiber, if I understand you right, is the unused capacity of the huge fiber optic cables carrying internet traffic. This would seem to support my argument that the "bandwidth" of the internet is not the crux of the problem.

This Moderator sees a lot of your points as so far off the mark at this point, he couldn't follow that yet another Topic needed to be started ... as so many points in this one still seem to be unresolved.  As far as insults, debate points, and such, I'd suggest that you started that with the above remark/question.

We seem to have two threads going now - "Is email authentication technically possible?", and "What are the implications for controlling spam?" This is difficult in an unthreaded forum, but I'll do my best. I'm not sure what you mean by my starting something. I (mistakenly) complained about what I thought were insults and debating tricks.

You keep repeating this as your mantra ... Please give me the "cheap" source for something like 100 contiguous IP addresses.

IP address are "cheap" to spammers, not to the folks who pay for them.

And once again "reputable" is in play ...???  For that matter, when's the last time you actually tried to register a "simple" Domain name?  Some Hosting services offer "free" Domain registration as part of the Hosting package, other ISPs offer $3 to $7 as a yearly fee dor a Domain name/registration ...  I am having a hard time going with your repeated "expensive" description for creating/registering Domain names.

Let's say I'm a spammer, and I've just registered 1000 new names. Since 1000 other spammers have also registered 1000 names, and all these names are rated "C" by default, they don't have much current value. We can spam all we want, but nobody is listening!! So how am I going to get some of these names up to a "B" rating where they will at least get through the initial block and to the spam filter, where I have a chance of fooling it with my latest "word salad"? I've got to get SpamCop, or some other company that puts out a widely used rating list, I've got to make them think I'm not a spammer. I can't apply under my own name, SpamCop has it in their database, so I find a friend who is willing to lie for me, and protect my identity in spite of a subpoena. I've done this fifteen times now, and I'm running out of friends. Also, SpamCop is getting very good at shutting down my domains, sometimes within hours of starting a spam run. After all that effort of getting a B rating, I can send only 100,000 ads for pen1s pilz, and the domain is slammed down to a "D" rating. D domains are worthless. I can send all the spam I want, and not 1 in 1000 will even accept my HELO. Even when I do get through, those 1000 other spammers are crowding me out. HELP!! I need some way to make this business profitable.

 

Your "impossible" word here also begs an argument.

Try sending me an email (dmq 'at' pobox.com ) with a forged address "aol.com". Let me know what happens. I think it will be impossible for you to forge the name "aol.com". AOL doesn't allow it.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0