What will it take to eliminate SPAM?

[InvisiBill]

I'm from /. =)

============================

Your post advocates a

(X) technical (X) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses

(X) Mailing lists and other legitimate email uses would be affected

( ) No one will be able to find the guy or collect the money

( ) It is defenseless against brute force attacks

( ) It will stop spam for two weeks and then we'll be stuck with it

( ) Users of email will not put up with it

( ) Microsoft will not put up with it

( ) The police will not put up with it

( ) Requires too much cooperation from spammers

(X) Requires immediate total cooperation from everybody at once

(X) Many email users cannot afford to lose business or alienate potential employers

( ) Spammers don't care about invalid addresses in their lists

( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it

(X) Lack of centrally controlling authority for email

( ) Open relays in foreign countries

( ) Ease of searching tiny alphanumeric address space of all email addresses

(X) Asshats

(X) Jurisdictional problems

(X) Unpopularity of weird new taxes

( ) Public reluctance to accept weird new forms of money

(X) Huge existing software investment in SMTP

(X) Susceptibility of protocols other than SMTP to attack

( ) Willingness of users to install OS patches received by email

( ) Armies of worm riddled broadband-connected Windows boxes

( ) Eternal arms race involved in all filtering approaches

(X) Extreme profitability of spam

( ) Joe jobs and/or identity theft

(X) Technically illiterate politicians

(X) Extreme stupidity on the part of people who do business with spammers

(X) Dishonesty on the part of spammers themselves

( ) Bandwidth costs that are unaffected by client filtering

( ) Outlook

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet none have ever been shown practical

( ) Any scheme based on opt-out is unacceptable

( ) SMTP headers should not be the subject of legislation

( ) Blacklists suck

(X) Whitelists suck

( ) We should be able to talk about Viagra without being censored

( ) Countermeasures should not involve wire fraud or credit card fraud

(X) Countermeasures should not involve sabotage of public networks

(X) Countermeasures must work if phased in gradually

(X) Sending email should be free

( ) Why should we have to trust you and your servers?

( ) Incompatiblity with open source or open source licenses

( ) Feel-good measures do nothing to solve the problem

( ) Temporary/one-time email addresses are cumbersome

(X) I don't want the government reading my email

( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.

( ) This is a stupid idea, and you're a stupid person for suggesting it.

( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

============================

What you're proposing would only work if everyone switched everything over to it at once. If everyone you interact with via email doesn't use it, it's worthless. Allowing emails from the "old system" to allow for people who haven't upgraded yet will allow spammers to do that as well. The upgrades would require massive software upgrades and probably some hardware upgrades too (actual changes to routers and stuff). I'm sure you've seen or heard about the problems that can arise from just from one Windows patch incompatibility on a few hundred PCs at some company. Now imagine everyone trying to upgrade every single mail server on the internet at once. I think it would be absolutely perfect for stopping spam (with the small side effect of stopping all other email too).

Also, whitelists suck. What you're proposing is a form of whitelist - you have to meet certain requirements to be in the "good" system. Maintaining credibility in this good system would most likely require a lot of effort (which means lots of money). Therefore, it would most likely cost to get into this good system. That right there discriminates against a lot of people. I don't have hundreds of dollars to spend on fees to "register" my domain/server/whatever with some authority just so that I can send an email to my friend asking him if he wants to get a pizza tonight.

Depending on the criteria used to let people into this system, there's also the issue of one-time spammers. Just like it's currently worth it for spammers to get an ISP account and spam until the account gets shut down, it will most likely be worth it for some spammers to register for this good system and spam until they get busted. Then they'll go somewhere else to get into the good system and do it again. If they're making enough money off it, they'll spend the money to get into the good system, even if only for a short while. Your system intends to make it easier to punish spammers a lot more severely, but I don't think it will stop spam completely (the same way that crime has yet to stop completely).

As others have said, packets do travel through a series of routers, hopping along the best path, but the message as a whole generally goes directly from the sending mail server to the receiving mail server. Since there's only one hop, RBLs that reject the connection already do exactly what you propose - if the sender is considered bad (or not considered good, however you want to look at it), the mail is rejected. That can be based on the ISP hosting the block, past spamming history, whether the mail server adheres to standards, etc.

You should really look into SPF. It does almost exactly what you described above. In your domain's DNS, you make a list of servers allowed to send mail for that domain. If someone gets a message claiming to be from your domain, but the sending mailserver isn't on the SPF list, it can be rejected as a forged message. Note that SPF only verifies the sender, it doesn't directly block spam. However, most spam is forged, and spam that isn't forged is easily tracked and stopped.

The two biggest problems with SPF are related to sending servers and forwarding. In the SpamCop case posted above, a SpamCop user could be sending from any SMTP server - their home ISP in New York, their vacation home in Florida, or from some business-related function in California. That's just one user - add up all those different options for every single SpamCop user, and you have an infinite number of servers where "SpamCop.net" email could originate. The proper solution to this is for SpamCop to provide an SMTP server. If SpamCop is allowing their name on emails, it seems like they'd want some control over who uses that name. With authenticated SMTP, they could make their SMTP server available to any of their users at any location, and all mail from SpamCop.net would come from one server (exactly the same way hosted email works). Add in an SPF record saying that mail sent from anywhere else claiming to be from SpamCop.net is forged, and they're pretty well covered.

Forwarding is a bit more complicated. If you have me[at]oldisp.net forwarding to me[at]newisp.net, the new mailserver will see all of the forwarded mail as being sent from the old mailserver, rather than from the server that actually sent the mail. This means that any SPF record which doesn't include the old ISP's mailserver would fail. The SPF people suggest that forwarding servers rewrite the email - that forwarded message would say it was from me[at]oldisp.net (which would pass SPF because it really is coming from the old ISP's mailserver), rather than from myfriend[at]isp2.net. SenderID/PRA attempts to do something like this without having to rewrite the message. Another option is to have oldisp.net check SPF on the sender (verify that it really came from myfriend[at]isp2.net), and configure newisp.net to ignore SPF for emails addressed to oldisp.net (since they would have already passed SPF at oldisp.net).

I think the present system works pretty well. With a few tweaks, I think it would work very well. The first step is moving away from the old model of allowing anyone on an ISP's network to send mail through the ISP's mailserver. Require authentication, just like every other service does. This allows the ISP to provide SMTP service to their customers regardless of their location, and block all unauthorized SMTP use in one sweep. I see no valid reason for having an open relay with authenticated SMTP options so readily available (except for the explicit purpose of anonymizing an email). Once we have all valid mailservers (public or private, free or paid) sending mail only for authentic users, simply block all open relays. With even just a simple weighting system, even the occasional false positive on a RBL should be able to send mail to you. Any spam coming from an authenticated user via a valid server would be just as easy to track and shut down as it would under the fancy new system.

[turetzsr]

I'm from /. =)

============================

Your post advocates a

(X) technical (X) legislative ( ) market-based ( ) vigilante

approach to fighting spam.

40502[/snapback]

...You appear to have "replied" to Miss Betsy's post 40486[/snapback] immediately above yours. If this was your intent, then I believe you completely misunderstand Miss Betsy. She has repeatedly, in other posts in these forums, displayed a preference against legislative approaches to spam control.

...Most of the rest of your post also seems to have little to do with Miss Betsy's post, which mentions

• the use of IP-based blacklists
  
• responsible use of the internet
  

[qjvgpuryy]

...You appear to have "replied" to Miss Betsy's post 40486[/snapback] immediately above yours. If this was your intent, then I believe you completely misunderstand Miss Betsy. She has repeatedly, in other posts in these forums, displayed a preference against legislative approaches to spam control.

...Most of the rest of your post also seems to have little to do with Miss Betsy's post, which mentions

• the use of IP-based blacklists
  
• responsible use of the internet
  

40504[/snapback]

I thought he was replying to PGTips91's post above Miss Betsy's post.

[Miss Betsy]

I think the present system works pretty well.

I didn't have time to read and assimilate Invisibill's post, but I agree with the above statement so I don't think he is replying to my post.

The only thing that is lacking in the present system is "Until the wider community of email users get involved in demanding..." the present system, it will not be as effective as it can be. And I think that is covered under "(X) Asshats"

Miss Betsy

[turetzsr]

I thought he was replying to PGTips91's post above Miss Betsy's post.

40505[/snapback]

...Could very well be but if you go into "Outline" view, it appears that he clicked the "Reply" button under Miss Betsy's post.

...Yet in PGTips91's post, I still don't see a suggestion for a legislative solution, just that e-mail users demand a solution.

[Miss Betsy]

...Could very well be but if you go into "Outline" view, it appears that he clicked the "Reply" button under Miss Betsy's post.

...Yet in PGTips91's post, I still don't see a suggestion for a legislative solution, just that e-mail users demand a solution.

40514[/snapback]

I think that PGTips91 proposed legislation in the beginning of the topic. He has to be replying to PGTips91 because he is agreeing with me that the present system works just as well as any new system would - especially if ISPs did not allow spam to go through open proxies and compromised machines. Invisibill must not have paid attention to what the different reply buttons do.

Miss Betsy

[PGTips91]

Executive Summary

The problems of spam, viruses, phishing and most email denial-of-service attacks can all be traced back to a single common cause

– lack of authentication in the email protocol SMTP.

And concludes with the following statement: --

With spam making up 80 percent of all email, massive credit card theft from phishing attacks and nonstop virus attacks, email is clearly broken.

Email authentication technologies offer a critical solution to fixing email. ...

These quotes are taken from none other than IronPort's own documentation, [pdf file at Email Authentication.]

I rest my case that the system is 'broken' and needs fixing - especially with proper authentication of the sender.

Looked at from another perspective, spam as a world-wide problem, is costing in the order of fifty billion dollars a year and rising.. Surely the answer must be to change the protocols so that Spammers are not given the chance to hide their identity and escape liability.

I don't accept the argument that "millions have been spent on looking for a solution, so if it could be done it would have been done already".

How much has really been done? In my research of the currently in-use and proposed protocols, I discovered to my amazement that the authors names are all listed - just a short list - and these have been around as RFC's since the late 90's. It would seem that everything is being tried without first changing the protocols so as to give them a much needed security layer! No wonder none of these proposals have worked very well or gained widespread acceptance.

Unfortunately most of the money spent looking for a solution has been invested in the hope of making more money from the situation rather than with the aim of everyone being better off equally. There simply hasn't been the will or the incentive to tackle the underlying problems in the antiquated protocols that are in use.

Maybe it would take a lot of people co-operating together and quite a bit of money to solve the problem, but at $50,000,000,000,000 per annum I think that, in normal circumstances, people would think it worthwhile spending quite a bit of money, time and effort in achieving that goal.

What is required is that the task of providing a solution be removed from competing players with their necessarily superficial approaches and the co-ordination of an industry-wide effort through an independent body with independent funding. This body would be commissioned with the task of speeding up the process of designing, testing and gaining wide implementation of a set of protocols capable of ensuring the continuance of email as a secure, efficient method of communication - without the burden of the parasitic infestation of Spammers eating out its vitals like tape-worms in a malnourished child.

There would be room in this model for an ongoing taskforce that would be charged with keeping the protocols ahead of any effort by the Spamming community to subvert the new protocols. There would be room for an advisory body to provide documentation, training and general education to enable the Internet community to cope with these necessary changes and to keep on top of the technicalities. After all, if Spammers are employing professional programmers to stay ahead in the present system it would be wise to keep the initiative with the wider Internet community by employing the same level of expertise to keep the protocols ahead of abuse in each new development in technology, as a public good shared by all. Since the cost-savings would vastly outweigh the funding required, everyone would be better off.

There are good models for this approach. As one example take IPv6 Forum at Home. There was a better example which I have lost the link to, but this will illustrate the point. By bringing together academics, industry leaders and technical people with a common goal, the task can be expedited. IPv6 needs to be implemented in the near future and this could be coordinated with that process.

Some radical new thinking and co-ordinated action is needed.

Paul

[PGTips91]

 

I think that PGTips91 proposed legislation in the beginning of the topic.  

Miss Betsy 

40521[/snapback]

 

I did not actually propose legislation in my initial post. That was inferred from my use of 'the Companies Office' as an illustration of the kind of verification needed to back up the commercial use of the Internet. Unless the identity of the person or organisation that you are dealing with is known or can be ascertained with certainty then normal legal processes cannot be used.

With proper identification, normal commercial laws and practice would be well able to make it uneconomic for Spammers to continue. Most providers already have provision in their Terms Of Use that could be used if the legal identity and address of the Spammer were known. Since most people have to pay for their Internet access, their provider would have to know their legal identity and address.

My call has been uniformly for the Internet community to cooperate in the matter of getting rid spam. I don't think that legislators can understand the problem well enough to solve it nor do I believe that they are independent enough of pressure groups to be unbiased.

Paul

[PGTips91]

The current Internet protocol, named Internet Protocol version 4 (or IPv4) is showing limitations now. To overcome these limitations, Internet Protocol version 6 (IPv6) was created and conceived as the successor of IPv4. IPv6 was developed by the Internet Engineering Task Force (IETF) since 1995 (http://www.ietf.org)

Whereas IPv4 makes available about 4 billion Internet addresses, IPv6 is able to handle over 340 trillion, trillion, trillion (or 3.4x10**38) addresses. In addition to solving the addressing problem, IPv6 also remedies to some other IPv4 shortfalls, such as the lack of "always-on" ability, consistent security, automatic configuration capabilities, and inefficient support for mobile nodes. In a nutshell, the benefits can be summarized, as follows: "always-connected" for billions of mobile devices, auto-configuration for ease of use, embedded security a pre-requisite for new applications in mobile banking, and so on. New applications are emerging such as use of Internet in public areas (schools, universities, airports), homes, cars, ships, aircrafts, which require that the involved devices are "always-on", and always reachable, will need a globally unique IP address, thus achievable only with IPv6.

Finally, "IPv4" does not anymore respond to civil society's new requirements.

New Internet (IPv6) Workshop

Just by way of illustration, the latest proposal for Internet Protocol, IPv6, has been under development since 1995!

Paul

[Wazoo]

Just by way of illustration, the latest proposal for Internet Protocol, IPv6, has been under development since 1995!

40533[/snapback]

Yes .... and??? Jusr as with the ARPANET / DARPANET 'experiment' .. this 'research' is also not open to the public, for all intents ... and even if public access were granted this morning, just how long would one wish to guess that the entire world will get / spend the money needed to upgrade all the servers, routers, controllers, and software to handle all this new traffic? I still deal with folks that feel good when they can get a 32k connection on their 56k modem, and that problem is based on POTS equipment still in use.

[InvisiBill]

Yeah, I'm used to VBB and phpBB, where there aren't 4 billion reply buttons. =) I seem to have used the one on the last post, rather than the general reply. I was indeed replying to the original poster.

I don't accept the argument that "millions have been spent on looking for a solution, so if it could be done it would have been done already".

Maybe it would take a lot of people co-operating together and quite a bit of money to solve the problem, but at $50,000,000,000,000 per annum I think that, in normal circumstances, people would think it worthwhile spending quite a bit of money, time and effort in achieving that goal.

What is required is that the task of providing a solution be removed from competing players with their necessarily superficial approaches and  the co-ordination of an industry-wide effort through an independent body with independent funding. This body would be commissioned with the task of speeding up the process of designing, testing and gaining wide implementation of a set of protocols capable of ensuring the continuance of email as a secure, efficient method of communication - without the burden of the parasitic infestation of Spammers eating out its vitals like tape-worms in a malnourished child.

There are good models for this approach. As one example take IPv6 Forum at Home. There was a better example which I have lost the link to, but this will illustrate the point. By bringing together academics, industry leaders and technical people with a common goal, the task can be expedited. IPv6 needs to be implemented in the near future and this could be coordinated with that process.

40531[/snapback]

Remember that the $50,000,000,000 (you have an extra set of zeroes in there) isn't $50B that's being spent on anti-spam, or the cost of bandwidth, or any specific thing like that. That's the figured cost of lost productivity. "In the U.S., for instance, spam's annual per-mailbox cost to businesses is $170." Figuring 2,000 work hours per year, that's 250 work days. spam costs 68 cents per day per user. Doesn't sound nearly so bad that way. They probably lose more money on extra coffee breaks and stolen office supplies.

You seem to be really stuck on changing the protocols. I agree that a lot of what we use on the internet today was designed in totally different times, and as such has some functional limits. However, short of fully associating every IP address with a full set of identifying ID (Americans value at least the perception of privacy, and most likely would never go for that), some new protocol isn't going to change anything. There will still be idiots who miscofigure computers, allowing others to take advantage of them. When that happens, do we blacklist that company's mail server from the internet for a month, or do we forgive them? If we blacklist them, they lose a ton of business - that's not really any incentive for them to use this new system. If we forgive them, we're not really any further ahead than the current system - again, not much incentive to upgrade every single mail server in the world.

Without any changes to most mail servers and clients, it's currently possible to have mail servers allow only authenticated users to send mail through them. I would think that tracing any sent email back to a user account, which is associated with billing information, would be just as identifying as any new system would reasonably be.

Again, without any major changes, you could configure your server to allow only "good" systems to send it mail. Blacklist known blocks of non-server IPs. Whitelist good senders. Blacklist known spammers. Whatever you prefer - but there are already ways to distinguish "good" servers from "bad" and handle their messages accordingly. Weighting systems like SpamAssassin even take the big picture into account, rather than automatically blocking all mail from a server because the version is before the name in the info or something stupid like that.

Personally, I think a large part of the problem is that people are choosing to use the old, simple, vulnerable ways rather than the newer, more secure ways. They think it's too much work to switch or that it will cause problems. In most cases it's easy and will be completely problem-free, but they need to be educated on that (and come to accept it). In some cases, people don't even realize that the old ways are bad, and that better ways are available.

The more "good" people do to keep things safe, the more "bad" stuff will stand out, making it easier to block.

This is nothing against you or anyone else, but did you know that regular email traffic, including your login name and password, are sent in plaintext? Most people know now that email is "like a postcard", but many don't realize that the whole transaction takes place in plaintext. Every time you use a standard email login, it's like shouting your login info over a speakerphone. All the authenticated SMTP servers I've seen do it over an encrypted connection. Your email still traverses the net in plaintext, but at least you're not giving out your login info, another bonus of SMTP auth.

[Miss Betsy]

The more "good" people do to keep things safe, the more "bad" stuff will stand out, making it easier to block.

Only many of the 'good' people need to be informed of what is safe and responsible so that they choose 'good and responsible' ISPs - instead of Comcast and Yahoo.

Miss Betsy