Jump to content

macquigg

Members
  • Content Count

    26
  • Joined

  • Last visited

Community Reputation

0 Neutral

About macquigg

  • Rank
    Member
  1. macquigg

    Why not whitelists

    There is a common and false argument I often hear from the anti-SPF crowd that adoption of SPF by spammers is proof that SPF is no good. Sorry if I misunderstood your innuendo, and sorry if my blunt criticism was taken as an insult. The topic of this thread is whitelists. Steve gave me some interesting data, which show that AOL is 100 times better at suppressing outgoing spam than either Sprint or Charter. See the "Domain Ratings" chart at http://www.ece.arizona.edu/~edatools/etc/ I've used this chart in discussions on other forums, always with the caveat that this is a very small sample. I would like to improve the chart with a better sample, maybe several hundred SpamCop reports, and maybe a better selection of "typical" domains to compare with AOL. I'll be happy to continue this discussion, but only if I get some useful information, and not a lot of sarcasm. -- Dave
  2. macquigg

    Why not whitelists

    You still have your head in the sand. SPF is *not* an anti-spam tool. If you want to learn about email authentication and how it relates to anti-spam tools, there are some references in the article you cited.
  3. macquigg

    Why not whitelists

    Take the number of reports for each domain, divide by the total volume from that domain, and use the ratio as a raw score. Then make a histogram of all the scores and decide where to draw the lines for each rating. Yes, it looks like you have plenty of raw data! All you need to do is process that data to get some meaningful, and easily understood results. -- Dave
  4. macquigg

    Why not whitelists

    Using these numbers and the total domain volumes from senderbase.org, we can derive a "spam Suppression Ratio" and compare the domains. I can't post a graphic here, but I put it at tinyurl.com/4gud4 Looks like AOL is 100 times better than the other domains. All good citizens, please use AOL :>) Absolutely right! Good domains like ISP will reduce their outgoing spam without any prodding from SPF. What SPF will do is push the other domains to clean up also.
  5. macquigg

    Why not whitelists

    Sorry for the confusion. It looked like we were done with the discussion on SPF and authentication technology in general. I think those domains were just pulled out of a hat, for illustration purposes, not to make any point that they were better. Are you seeing any differences, on average, between good domains and bad, or is the spam pretty much coming evenly from everywhere? Does AOL, for example, have lower than average? They claim to have eliminated their outgoing spam. "We do not send it any more." -- AOL postmaster Carl Hutzler http://www.circleid.com/article/917_0_1_0_C/ It would be interesting to see some actual numbers of spam reports from AOL compared to other well-known domains.
  6. macquigg

    Why not whitelists

    I just came across an wonderful piece of art and explanation on how things will work a year from now, assuming email authentication is widely adopted. http://spf.pobox.com/aspen.html Looks like SpamCop might be in a good position to participate - lots of subscribers worldwide, an up-and-running real-time feedback system, a good reputation in the business. -- Dave
  7. macquigg

    Why not whitelists

    OK, we now have what I think is a correctly set up forwarder, bmsi.com. Here is a telnet session trying to fake 'amazon.com' in a mail to 'dmq at bmsi.com'. 220 mail.bmsi.com ESMTP Sendmail 8.13.1/8.13.1; Tue, 22 Feb 2005 00:24:02 -0500 HELO smtp.egreinsch.com 250 mail.bmsi.com Hello egrrtr [141.156.111.136], pleased to meet you MAIL FROM: <forged at amazon.com> 550 5.7.1 <forged at amazon.com>... SPF fail: see http://spf.pobox.com/why.html And here is the top header on a correctly addressed email from gain.com to 'dmq at bmsi.com': Return-Path: <SRS1=UPSe=bmsi.com==6Vba5=RE=gain.com=dmq at bounce2.pobox.com> Notice I used two forwarders: bmsi.com, then pobox.com Looks like SRS works if it is set up correctly. The problem is social engineering. The warring camps are not talking to each other. To address this problem, I started a discussion on the SPF mailing list v2.listbox.com/200502/index.html]http://archives.listbox.com/spf-discuss[at]v2...0502/index.html "Email Forwarder's Protocol (EFP)". -- Dave
  8. macquigg

    Why not whitelists

    If a postmaster ignores a flood of messages about authentication failure, he won't be able to send mail to *any* receiver that does authentication. That leaves the spam category. If the message is rejected because it is from a known spamming domain, you probably don't want to reply anyway. They will know very well why their messages are being ignored. For messages that get past authentication, and past the "pure spam domain" blocklist, then we have a situation much like today, but with most of the worst spam already discarded. The options include everything we do now, (report to SpamCop, etc.) but with the additional information on domain rating, and with the option to reply to the postmaster at the actual domain that sent the message. Nothing is taken away. We just have some very useful new options. -- Dave
  9. macquigg

    Why not whitelists

    Authenticate: SPF1 [<IP Address>] <senders-domain> PASS Let's make a distinction between the author and the sender. The Authenticate: header contains only informtion that has been authenticated. There is no way to authenticate the author's name, as that is known only within the sender's domain. The bounce should go to postmaster[at]<senders-domain> The postmaster can then decide whether to use the information in the From: header to forward the bounce back to the author. Most bounces will be for problems the domain owner should take care of ( authentication failure, possible spam, etc.) It looks to me like the ball has been dropped on this one last vital piece to make SPF a fully functional system that can be used even with forwarders. DomainKeys avoids the problem by going "end-to-end" with a digital signature, but I don't know if computational costs are an issue. We really do need a standard that will allow forwarders to participate in an IP-authenticated transfer. The standard should allow for different authentication protocols (SPF1, SPF2, SenderID, etc.). The captured IP address should be displayed, even if it is only a temporary address. Then we should show the domain name, determined by whatever method the protocol calls for. Finally, we need the result of the authentication on that name, using whatever keywords are specified in the protocol. After these four standard parts can come any number of free-form parameters ( hash code, time stamp, etc.) These need not be standardized, because they serve only as a "cookie" for the forwarder's own processes. -- Dave
  10. macquigg

    Why not whitelists

    The key difference is that with the current system, list providers like SpamCop have to react quickly every time a spammer gets a new IP address. Instead, the burden should be on domain owners, who will have to prove to SpamCop that their domain is *not* a source of spam. Only those domains that are serious about operating public mail servers will bother to get rated. -- Dave
  11. macquigg

    Why not whitelists

    With authentication, it should not be necessary to bounce during the same SMTP session. As long as the authenticated address is saved, a bounce can go to that address sometime later. Regardless of whether you use telnet or DHCP, or any other protocol, when you establish an SMTP session, the receiver should authenticate your domain and make sure that the IP address on the incoming packets is an authorized address for that domain. Seems to me the domain to be authenticated should be whatever is used in the HELO command that requests the SMTP session. All others should be irrelevant. I still think this could be done more clearly by having the forwarder just prepend a header: Authenticate: SPF1 [<IP Address>] <senders-domain> PASS Then when a bounce comes back, the forwarder just needs to peel off the top Authenticate line and use that to forward the bounce. I understand the forwarder cannot trust what comes back from the receiver, but that could be verified by just having the forwarder store a hash code for every forwarded message. If a fresh hash on a bounced message doesn't match anything forwarded in the last 24 hours, don't forward the bounce. http://en.wikipedia.org/wiki/Email_Authentication I don't understand the need for multiple forwarders in series. I have two, but they both forward to my current ISP. Nevertheless, authentication should work in your situation also. As I understand it, SRS saves only the first sender's domain, and not that of any previous forwarders. This seems terribly insecure to me. There needs to be an authenticated domain for each forwarder. Your test message with the forged amazon.com got through: Return-Path: <SRS0=ekIn=RC=amazon.com=forged at bounce2.pobox.com> Seems like SRS is broken ( at least at pobox.com )!! That surprises me, because these are the folks pushing SPF/SRS. I'll ask them about this incident, and post the result here later. Your telnet test looks good. That should allow you to fake anything, including the SMTP "envelope" info. -- Dave
  12. macquigg

    Why not whitelists

    This *is* disturbing. I don't understand why SRS needs to be so complex. Seems to me that a forwarder like pobox.com could convey its authentication of the sender by adding a simple unambiguous header: Authenticate-SPF: [<IP Address>] <senders-domain> VERIFIED ( or FAILED as the case may be). Seems to me that a bounce should go back the same path it came, not to some header address that might be forged. See the section "Responsibility of Forwarders" in http://en.wikipedia.org/wiki/Email_Authentication Yes, it came through. This time not to my spam bucket. Here is the report: spam: ------------------------ Spamnix spam Report ------------------------- spam: Spamnix identified this message as spam. This report shows which spam: rules matched the message and how many points each rule contributed. spam: spam: Content analysis details: (5.4 hits, 5.0 required) spam: 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% spam: [score: 100.0%; H*p:D*com:99 H*r:HELO:99] spam: [H*c:charset:99 N:H*r:N.NN.NN:99 H*c:plain:99] spam: [H*c:us-ascii:99 N:H*r:NNN.NNN.NN:99] spam: [N:H*r:NN.NNN.NN:99 H*r:8.12.11:99 Virus:99] spam: [N:H*x:N.N:99 H*x:Windows:99 Release:99] spam: [N:H*m:NNNNNNNNNNNNNN:2 N:H*r:sk:cpe-NN-:98] spam: [H*x:Version:98 H*F:D*aol.com:98 HTo:U*dmq:97] spam: [Database:96 H*c:flowed:96 H*x:QUALCOMM:96] spam: [Anti-Virus:96 H*x:Eudora:96 H*c:format:96] spam: [N:N.N.NNN:96 N:HMime-Version:N.N:94] spam: [HMime-Version:1.0:94 test:6 virus:92 H*m:com:12] spam: [checked:86] spam: spam: spam level: ***** spam: --------------------- End of Spamnix spam Report --------------------- Looks like it saw some inconsistency in the headers, but I can't really follow this. Try the same test, but forging 'amazon.com'. Unlike aol.com, Amazon's SPF records indicate that only Amazon's servers are authorized senders for amazon.com. It will be interesting to see how pobox.com handles the forwarding. -- Dave
  13. macquigg

    Why not whitelists

    Oops !! I just checked the SPF records for aol.com, and they *don't* prohibit other servers from using their name!! The last item in their list of valid senders is "?all", which means (as I understand it) *no policy* on other servers. This is some kind of interim setting, I suppose to allow others to look at aol's soon-to-be-enforced SPF records, and get ready for them to change the last item to "-all", which means "no other servers allowed". Please try the same test with amazon.com. They have a definite "-all" at the end of their record. -- Dave P.S. If you want to check the SPF records for any domain go to http://mxtoolbox.com/spf.aspx
  14. macquigg

    Why not whitelists

    Which address did you forge? SPF checks the envelope, not any of the headers. Does spamcop.net check SPF records on incoming mail? I did receive your email. See headers below. It went straight to my spam bucket, with 99% certainty, but it shouldn't have gotten that far if the envelope was forged. If not, you haven't really hidden your true domain. -- Dave Return-Path: <SRS0=1WH5=RB=aol.com=forged<at>bounce2.pobox.com> Delivered-To: dmq<at>gainusa.com Received: ( **SaniMail 89778 invoked from network** ); 19 Feb 2005 20:27:37 -0000 Received: from unknown (HELO lime.pobox.com) (208.58.1.198) by mail5.mailsystem.us with SMTP; 19 Feb 2005 20:27:37 -0000 Received: from lime.pobox.com (localhost [127.0.0.1]) by lime.pobox.com (Postfix) with ESMTP id 810311B9A83 for <dmq<at>gainbroadband.com>; Sat, 19 Feb 2005 15:34:12 -0500 (EST) Delivered-To: dmq<at>pobox.com Received: from mxsf18.cluster1.charter.net (mxsf18.cluster1.charter.net [209.225.28.218]) by lime.pobox.com (Postfix) with ESMTP id 56B5D1B9A2E for <dmq<at>pobox.com>; Sat, 19 Feb 2005 15:34:11 -0500 (EST) Received: from mxip01.cluster1.charter.net (mxip01a.cluster1.charter.net [209.225.28.131]) by mxsf18.cluster1.charter.net (8.12.11/8.12.11) with ESMTP id j1JKYA43004461 for <dmq<at>pobox.com>; Sat, 19 Feb 2005 15:34:11 -0500 Received: from cpe-24-177-39-162.ma.charter.com (HELO steven.aol.com) (24.177.39.162) by mxip01.cluster1.charter.net with ESMTP; 19 Feb 2005 15:34:09 -0500 X-Ironport-AV: i="3.90,100,1107752400"; d="scan'208"; a="580630140:sNHT3090204222" Message-Id: <5.1.0.14.2.20050219153320.00b80ec0<at>wheresmymailserver.com> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sat, 19 Feb 2005 15:34:06 -0500 To: dmq<at>pobox.com From: Steven Underwood <forged<at>aol.com> Subject: AOL test Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed; x-avg-checked=avg-ok-3AD5650E *Moderator edited email addresses*
  15. macquigg

    Why not whitelists

    It looks like I miscommunicated. I apologize for assuming the responses were merely "scoring points". I'm seeing a fundamental difference between the bandwidth of the radio spectrum, and the "bandwidth" of the internet, a difference that has profound implications for the question of what to do about spam. Maybe we should just leave it at that. Dark fiber, if I understand you right, is the unused capacity of the huge fiber optic cables carrying internet traffic. This would seem to support my argument that the "bandwidth" of the internet is not the crux of the problem. We seem to have two threads going now - "Is email authentication technically possible?", and "What are the implications for controlling spam?" This is difficult in an unthreaded forum, but I'll do my best. I'm not sure what you mean by my starting something. I (mistakenly) complained about what I thought were insults and debating tricks. IP address are "cheap" to spammers, not to the folks who pay for them. Let's say I'm a spammer, and I've just registered 1000 new names. Since 1000 other spammers have also registered 1000 names, and all these names are rated "C" by default, they don't have much current value. We can spam all we want, but nobody is listening!! So how am I going to get some of these names up to a "B" rating where they will at least get through the initial block and to the spam filter, where I have a chance of fooling it with my latest "word salad"? I've got to get SpamCop, or some other company that puts out a widely used rating list, I've got to make them think I'm not a spammer. I can't apply under my own name, SpamCop has it in their database, so I find a friend who is willing to lie for me, and protect my identity in spite of a subpoena. I've done this fifteen times now, and I'm running out of friends. Also, SpamCop is getting very good at shutting down my domains, sometimes within hours of starting a spam run. After all that effort of getting a B rating, I can send only 100,000 ads for pen1s pilz, and the domain is slammed down to a "D" rating. D domains are worthless. I can send all the spam I want, and not 1 in 1000 will even accept my HELO. Even when I do get through, those 1000 other spammers are crowding me out. HELP!! I need some way to make this business profitable. Try sending me an email (dmq 'at' pobox.com ) with a forged address "aol.com". Let me know what happens. I think it will be impossible for you to forge the name "aol.com". AOL doesn't allow it.
×