Jump to content


  • Content Count

  • Joined

  • Last visited

Posts posted by PeterJ

  1. I have read good things about XWall, but have no experience with it. It has an amazing feature set for $400 and the licensing is not based on the # of users. Take a look at the change log and you will see that the author is quick to update the program with new features...for example XWall already has capabilty for SURBLs and SPF.

    Again, I have never used it, so do not take my word, research it if you are interested.


  2. Great example. That is odd that no URIBL tests tripped from SpamCop's implementation of SA as last I knew we are using them...

    It would be great if JT took a look at this and could confirm no issues with SpamCop's setup. I took a look at some of my recent spam and I have seen URIBL tests being tripped at least in a couple spam messages from blades 1,2,3,4 , and 6. I have not seen a recent message from blade 5 with URIBL tests being tripped, but this could just be a conincidence. Anyone else see any patterns?


  3. Here is a list of the default tests performed by SA 3.0. Bear in mind that JT could have modified scoring on any of these and additional tests could have been added.


    My guess is that if you posed your question to SpamAssassin administrators and programmers in general they would expect messages such as the one you have posted to get caught by bayesian scoring that would have tipepd the scale up to 4 or 5 total points.

    Keep in mind that SpamCop's implementation of SA includes neither blacklists nor bayesian techniques therefore this is irrelevant.

    Long story short is that SpamCop is harnessing some of the power that SA allows for and it is still quite effective. Since it is unlikely that the bayesian or blacklist portions of SA will be added on...if the SpamCop's filtering is not adequate for you, I recommend taking matters into your own hands and using a bayesian filter or some other kind of client filter to help with the few emails that are not getting caught. As you will find elsewhere I am a strong believer in POPFile


  4. In case anyone cares to check out the two recent announcements regarding the SpamAssassin 3.0 release they are pasted below.

    The following quote is from Daniel Quinlan and can be seen in context here:


    Apache Software Foundation Announces SpamAssassin 3.0 Release

    Forest Hill, MD - September 22, 2004 -- The Apache Software Foundation is

    pleased to announce the release of SpamAssassin 3.0.  SpamAssassin 3.0

    contains a number of new technologies designed to protect against the

    changing techniques used by spammers.  This is the first SpamAssassin

    release as an Apache Software Foundation project and under the Apache

    License.  The release is available from the Apache SpamAssassin web site

    (http://spamassassin.apache.org/) via the Apache mirror network.

    SpamAssassin 3.0 delivers many new features including support for sender

    authentication using the Sender Policy Framework (SPF), checking for web

    links of known spam advertisers, a modular plugin architecture, improved

    SQL database support for storing user data in server installations, and

    improved email classification.

    SpamAssassin's practical multi-technique approach, modularity, and

    extensibility continue to give it an advantage over other anti-spam

    systems.  Due to these advantages, SpamAssassin is widely used in all

    aspects of email management.  You can readily find SpamAssassin in use in

    both email clients and servers, on many different operating systems,

    filtering incoming as well as outgoing email, and implementing a very

    broad range of policy actions.  These installations include service

    providers, businesses, not-for-profit and educational organizations, and

    end-user systems.  SpamAssassin also forms the basis for numerous

    commercial anti-spam products available on the market today.

    About SpamAssassin

    SpamAssassin is an intelligent email filter which uses a diverse range of

    tests to identify unsolicited bulk email, more commonly known as "spam".

    These tests are applied to email headers and content to classify email

    using advanced statistical methods.  In addition, SpamAssassin has a

    modular architecture that allows other technologies to be quickly wielded

    against spam and is designed for easy integration into virtually any email


    About the Apache Software Foundation

    The Apache Software Foundation provides organizational, legal, and

    financial support for a broad range of open source software projects.  As

    a US 501©(3) public charity, the Foundation provides an established

    framework for contributions of both intellectual property and funding for

    the support of open source software development.  Through a collaborative

    and meritocratic development process, Apache projects deliver

    enterprise-grade, freely available software products for the public

    benefit, attracting large communities of users and enabling future

    innovation, both commercial and individual, through its pragmatic Apache


    Press Contact:

      press <at> apache.org


    Daniel Quinlan                  ApacheCon! 13-17 November (3 SpamAssassin

    http://www.pathname.com/~quinlan/  http://www.apachecon.com/  sessions & more)

    The following quote is from Daniel Quinlan and can be seen in context here:


    (technical information about SpamAssassin 3.0.0 release)

    SpamAssassin 3.0.0 is released!  SpamAssassin 3.0.0 is a major update and

    includes a number of new email and anti-spam technologies.

    SpamAssassin is a mail filter which uses advanced statistical and

    heuristic tests to identify spam (also known as unsolicited bulk email).



      MD5 checksums

      SHA1 checksums

      GPG key information

      Major feature list

      Important installation notes

      New rules

      Spamd changes

      Engine changes



      Please use a mirror off of <http://spamassassin.apache.org/>.

    MD5 checksums:

      b77c7b29ddc4283d597610bf540670d9  Mail-SpamAssassin-3.0.0.tar.bz2

      e38035c260310e18158d95a41cadae93  Mail-SpamAssassin-3.0.0.tar.gz

      7b5d10c33d33e16a849fdba7b2c9bc43  Mail-SpamAssassin-3.0.0.zip

    SHA1 checksums:

      d722c0e27b4b9c8117e5c583c0924a0b81f62a30  Mail-SpamAssassin-3.0.0.tar.bz2

      d52c317483483874f2b7e5dd544094249bedbdad  Mail-SpamAssassin-3.0.0.tar.gz

      3a0c3373b2655c14c14ababa61b7c3791340ee72  Mail-SpamAssassin-3.0.0.zip

    GPG key information:

      The release files also have a .asc accompanying them.  The file serves

      as an external GPG signature for the given release file.  The signing

      key is available via the wwwkeys.pgp.net key server, as well as a file

      named GPG-SIGNING-KEY in the download directory.

      pub  1024D/265FA05B 2003-06-09 Apache SpamAssassin <release <at> spamassassin.org>

        Key fingerprint =3D 26C9 00A4 6DD4 0CD5 AD24  F6D7 DEE0 1987 265F A05B

    Major feature list:

      - SpamAssassin is now part of the Apache Software Foundation and has an

        improved software license, the 2.0 version of the Apache License.

      - SpamAssassin now includes support for SPF (the Sender Policy

        Framework, http://spf.pobox.com/).

      - Web site links contained in the message are checked against SURBL and

        SBL.  SURBL and SBL track sites that advertise with spam, known spam

        sources, and spam services.

      - The new 3.0 architecture allows third-parties to easily add plugin


      - There is now SQL database support for both the Bayes and

        auto-whitelist modules, allowing more large sites to easily deploy


      - A more accurate simulation of email client handling of MIME and HTML

        improves our accuracy.  In addition, there is better detection and

        handling of spammer techniques that try to trick anti-spam software.

    Important installation notes:

      - The SpamAssassin 2.6x release series was the last set of releases to

        officially support perl versions earlier than perl 5.6.1.  If you are

        using an earlier version of perl, you will need to upgrade before you

        can use the 3.0.0 version of SpamAssassin.

      - SpamAssassin 3.0.0 has a significantly different API (Application

        Program Interface) from the 2.x series of code.  This means that if

        you use SpamAssassin through a third-party utility (milter, etc,) you

        need to make sure you have an updated version which supports 3.0.0.

      - The --auto-whitelist and -a options for "spamd" and "spamassassin" to

        turn on the auto-whitelist have been removed and replaced by the

        "use_auto_whitelist" configuration option which is also now turned on

        by default.

      - The "rewrite_subject" and "subject_tag" configuration options were

        deprecated and are now removed. Instead, using "rewrite_header Subject

        [your desired setting]".  e.g.

          rewrite_subject 1

          subject_tag ****spam(_SCORE_)****


          rewrite_header Subject ****spam(_SCORE_)****

      - The Bayesian storage modules have been completely re-written and now

        include Berkeley DB (DBM) storage as well as SQL based storage (see

        sql/README.bayes for more information).  In addition, a new format has

        been introduced for the bayes database that stores tokens in fixed

        length hashes.  All DBM databases should be automatically converted to

        this new format the first time they are opened for write.  You can

        manually perform the upgrade by running "sa-learn --sync" from the

        command line.

        The "sa-learn --rebuild" command has been deprecated; please use

        "sa-learn --sync" instead.  The --rebuild option will remain

        temporarily for backwards compatibility.

      - "spamd" now has a default max-children setting of 5; no more than 5

        child scanner processes will be run in parallel.  Previously, there

        was no default limit unless you specified the "-m" switch when

        starting spamd.

      - If you are using a UNIX machine with all database files on local

        disks, and no sharing of those databases across NFS filesystems, you

        can use a more efficient, but non-NFS-safe, locking mechanism.  Do

        this by adding the line "lock_method flock" to the

        /etc/mail/spamassassin/local.cf file. This is strongly recommended if

        you're not using NFS, as it is much faster than the NFS-safe locker.

      - Please note that the use of the following command line parameters for

        spamassassin and spamd have been deprecated and are now removed.  If

        you currently use these flags, please remove them:

          in the 2.6x series: --add-from, --pipe, -F, -P, --stop-at-threshold, -S

          in the 3.0.x series: --auto-whitelist, -a

      - The following flags are deprecated and will be removed in a future

        major release: --whitelist-factory, -M, --warning-from, -w,

        --log-to-mbox, -l.

      - SpamAssassin runs in "taint mode" by default for improved security.

        Certain third-party modules, such as Razor v2, may be incompatible

        with taint mode. For Razor v2, you will need to be using v2.40 of

        razor-agents or higher which allows taint mode by default.  Earlier

        versions which are patched to allow taint mode may be used as well.

      - Finally, 2.6x deprecated the use of the "check_bayes_db" scri_pt, and

        it is now no longer available.  Please see the sa-learn man/pod

        documentation for more info.

    New rules:

      - SPF is supported using the Mail::SPF::Query module.

      - There are new rules and code to combat Bayes-poisoning text and random

        hash-busters; Habeas rules now check against the Habeas user list to

        combat forged marks used in spam.

      - URIDNSBL rules: these do DNSBL lookups on URLs using SURBL and SBL,

        allowing spamvertised URLs found in the message body to be scored.

      - Spamhaus XBL and a variety of new DNSBL rules were added.

      - Hashcash is supported.

      - Bob Menschel's "longwords" rules were added.

      - The "backhair" rules were added to combat HTML obfuscation, a

        technique based on Jennifer Wheeler's "backhair" ruleset.

      - Matt Kettler's "antidrug" rules were added.

      - Anti-fraud rules from Matt Yackley were added.

      - Some hostname-based blocklist tests based on the envelope sender

        address were added.

      - And more...

    Spamd changes:

      - spamd now uses a "preforking" model instead of "fork per message".

      - There is a new log format, detailing message-id, resent-message-id,

        tests hit, autolearn status, and other data in a format compatible

        with mass-check to provide more information for spamd log-summary


    Engine changes:

      - Plugins: third-party modules can now be written and loaded dynamically

        from inside SpamAssassin, to provide support for entirely new tests

        and functionality.

      - SQL support for Bayes and AWL storage was added, thanks to Michael

        Parker.  See sql/README.bayes and sql/README.awl for additional


      - There was a ground-up rewrite of the MIME parser.  It now deals

        correctly with complex MIME structures, including entire

        message/rfc822 message attachments.

      - Rules can now test the "MAIL FROM:" address used in the SMTP

        transaction if it was logged to the message headers using the

        "EnvelopeFrom" pseudo-header.  This allows rules such as SPF to be


      - Added an optional fast Bayes locking mechanism for local usage only

        (it's not NFS safe) using "lock_method flock".

      - Support for parsing mbx mailboxes (as used by UW IMAP) was

        added.  Thanks to John Newman for this patch.

      - There is a refactored configuration parser to split parser code from

        the configuration settings.

      - Bayes databases can now be backed up and restored using --backup and


      - Config files can now include other files using the "include" command.

      - The GA-based evolver was replaced with a fast Perceptron score

        generation tool by Henry Stern; scores can now be generated much more


      - The "spamassassin" scri_pt can now check collections of mail en masse.

        This allows commands such as "spamassassin -d --mbox file1" to run

        over an entire mbox file.  It also works for checks, adding to

        white/black-lists, etc.

      - The Windows support improved.


      - A Dutch translation was added, thanks to Jesse Houwing.

      - A French translation was added, thanks to Michel Bouissou.

      - A German translation was added, thanks to Klaus Heinz.

      - A Polish translation was added, thanks to Jerzy Szczudlowski and radek

        at alter dot pl.


    Daniel Quinlan                  ApacheCon! 13-17 November (3 SpamAssassin

    http://www.pathname.com/~quinlan/  http://www.apachecon.com/  sessions & more)

  5. Thanks for updating SpamAssassin JT! I know that many SpamCop mail users will appreciate the significant improvements that the SpamAssassin team have made in version 3.0.

    My favorite improvement is SURBL checking and it is fitting that SpamCop mail users who report websites in spam via SpamCop will now directly benefit from this action by contributing to at least one SURBL.

    A suggestion to SpamCop mail users who have their SpamAssassin thresholds set very low (at 1, 2, or 3 perhaps), I believe that it will be wise to raise your threshold up to 4 or 5 since SpamAssassin 3.0 will be more effective than you were used to. You may find that with a higher setting for your SpamAssassin, you can still effectively filter the spam while hopefully not having to whitelist.

    I think I will test the new SA out and turn off my other mail filtering and sorting for the time being.

    Way to be on top of upgrading JT, afterall it was only released yesterday!

  6. Not "tuning", "turning on"...as in choose to use SURBL checks in SpamCop's SA implementation. This can be done with either the current version that I assume SpamCop is still running or with the soon to be released SA version 3.

    Note that I am not suggesting that JT turn on DNSRBL checking internal to SpamCop's SA, nor am I rehashing the whole bayesian thing, only suggesting that adding SURBL checks to SpamCop's current SA scoring would be beneficial. SpamCop mail users would not be providing any feedback in regards to this setup in the form of "tuning" or "training."

    Also, as noted elsewhere, JT likes SA rules/tests that have very low false positive rates and SURBL tests so far have fallen in this category.

    Link to SURBL info

  7. Your two messages are good examples of a SpamAssassin implementation at your provider that is more optimally setup when compared to SpamCop's implementation of SA. Note that the differences in the scores are drastic and this is because of several reasons. On the first message your provider is using both DNSRBL and SURBL checks to contribute to the overall SpamAssassin score. On the second message perhaps they are still using both, however it only tripped the SURBL checks and not the DNSRBL ones.

    Thanks for posting these messages they help confirm my feeling that SpamCop's SA implementation is not optimal. In a recent thread on the topic of SURBLs I asked JT to comment on the possibility of setting SURBL checks in place, however he rarely makes an appearance in the forums these days.

    Our best bet for JT improving SpamCop's SA is when SpamAssassin 3 is officially released and then ask him to consider implementing this *with* turning on the SURBL checks. I think SpamAssassin 3 is currently at RC4, so I imagine it cannot be too much longer before it is officially released.


  8. Microsoft's was Caller ID, but reading up at http://www.microsoft.com/senderid, indicates that Caller ID, SPF, and Submitter Optimization were rolled into "Sender ID Framework" prior to submission to the IETF.

    It was interesting to read what is going on with discussion on the licensing for the "Sender ID Framework", here is a paragraph from the following article: http://www.internetnews.com/xSP/article.php/3399421

    But the combination created some new problems along the way. SPF is popular with open source MTA's (define) like Sendmail, Postfix, Qmail and Exim, which license the software under various open source licenses. Caller ID for E-Mail, however, comes with a license that's royalty-free but contains clauses that have raised questions since the the two technologies merged.

    Figures once MS got invovled the licensing would go to hell...

  9. I am not exactly sure what issue you are describing, but I am going to guess that you cannot successfully load websites after clicking on hyperlinks in your tesco webmail...

    If your question was how to get into you mail to begin with then go to:


    Otherwise the following Microsoft Knowledge Base articles may be useful:

    Links not properly opening from Outlook Express

    Links not properly opening from Internet Explorer

    Also, since you recently performed a system restore it would likely be wise to make use of the Windows Update site to ensure that your computer is updated properly.

    Welcome to the SpamCop web forums,


  10. I forgot to inlcude a quote from JT (Feb 17, 04) in these forums:

    I probably don't have a problem with those. I'll look into that. That said, it's my intention that the rules we use are very safe. We use the standard rules, plus Big Evil. Big Evil is intended to be a zero-false-positive list. It's all URLs and if you show legit email with a URL in it, he'll remove the URL from Big Evil.

    This was part of his response to a request for improving SpamCop's SpamAssassin implementation. I wanted to bring this back up because it is very relevant to the topic of SURBLs. "Big Evil" was great while it lasted, I nice list of spammy URLs that SpamAssassin could check against and Chris Santerre (of SA fame) did a great job of manually maintaining the list. As JT mentions above, one had to email Chris and request that a legitimate URL be removed from the list and then every time an updated list would come out SpamAssassin administrators would then need to incorporate this. It becomes pretty clear that over the long haul this became too cumbersome...guess what replaced it? SURBLs! At least one of the SURBLs that Jeff Chan and company have developed has absorbed the "Big Evil" list. Just wanted to bring this up as further evidence as to why SpamCop should implement the use of a SURBL in the SpamCop SpamAssassin implementation. My logic on this as follows: JT liked and was using "Big Evil", JT likes "very safe" rules, "Big Evil" is no longer being developed and has been absorbed by a SURBL, therefore JT should look into implementing a SURBL to maintain SpamCop's SpamAssassin efficacy.

    JT? (I know you have not posted in the forums for months probably, but if you read this it would be great to know your thoughts.)

  11. To those who have recently posted to this thread, if you have a sincere interest in Jeff Chan's initial idea, please note that he first posted regarding this quite a while ago. Since that time it is no longer an idea, it is real. Jeff Chan and others have developed the means to handle redirection sites, not sure about the use of unicode. SURBLs (spam URI Realtime Blocklists) have proven quite effective when combined with traditional blocklists, bayesian filtering, and other types of spam detection/scoring.

    I mention this only because I have not seen Jeff Chan around these forums for many months and would be surprised of his accidental return unless someone tells him his original thread is still alive. If you wish to reach him, his website on this topic has info on SURBLs, his email address, and available surbl mailing lists. Check it all out at: http://www.surbl.org/

    A question for JT: Can we please implement URI checking in SpamCop's SpamAssassin setup? (Note that this will be included with v3.0 of SpamAssassin, soon to be released) If you are waiting for SA 3.0 to come out before trying this, I understand.

  12. I'll throw in my current usage of SpamCop for a contrast. I have had a SpamCop email account for around 4 years by now and have changed ISPs and moved once, so for starters having my primary email address (with SpamCop) separated from those who provide me internet service is a big draw for me. I typically forward whatever addresses I might have with my ISP at the time to my SpamCop mail account. I give out my SpamCop mail address all the time to friends and companies that I have relationships with. If I am skeptical of a place that wants an email addy for me, then I use one I created with my ISP for this purpose.

    Filtering and Whitelist setup:

    I have turned off all blacklists and SpamAssassin so that SpamCop is not doing any filtering for me at all. Also my whitelist is completely empty and I am not using it. Instead I am relying on POPFile to sort my mail for me into several folders. I am running the latest POPFile CVS which has support for IMAP accounts. POPFile is a bayesian based email sorting tool that allows me great flexibility when coupled with my SpamCop IMAP access. My installation of POPFile runs on my file server at home such that it runs 24x7 always sorting my mail as soon as it sees something new in my SpamCop account (via IMAP.) The advantages of this are excellent, because it means that none of my workstation computers are responsible for sorting, it all happens on my backend. It essentially gives me server side filtering (although my server) for my SpamCop mail account, a feature that many SpamCop users have requested over time. POPFile's support for IMAP remains young, but I have assisted with reporting bugs when I experience them, overall it is a great program. I have not tracked my filtering stats all that well but usually I seem to be at a 98% accuracy rating with POPFile's classifications.

    Reading Mail:

    I typically use Thunderbird to read my mail utilizing IMAP when at home, otherwise I use the SpamCop webmail interface (Horde IMP.) The glory of my POPFile setup is that regardless of where I check my mail, either at home with a client, or away with Horde, my mail is always sorted for me when I go to look at it.

    Reporting spam:

    I have found myself almost exclusively using the VER for reporting because it allows me to easily see the email addy and the subject and determine whether it is spam or not. If I use the webmail interface I find that I "miss" the email address and that the "Display Name" is not useful, sometimes even a hindrance. So, I use quick reporting from VER and report the 20 to 30 spam I get per day with this method. If I get busy and I do not make it to VER then I usually just delete messages older than a day before resuming my quick reports again. I do look through the "Quick Report" email repsonses I get back from SpamCop to primarily ensure I have not reported myself, but secondarily for mistakes that standout.

    Training my POPFile filtering:

    This is the easy part and another great thing about POPFile's IMAP support. When I get either a false positive or a false negative, simply moving the message to the correct folder (Inbox >> Held Mail or Held Mail >> Inbox) causes POPFile to reclassify the message and become aware of the mistake it made. Another benefit of POPFile being setup and running 24x7 on a server at my residence combined with IMAP support means that I can TOE (train on errors) when I am away from my residence and only using SpamCop's webmail acccess. All I have to do in SpamCop's webmail is move the mistakenly classified messages to their correct folder.


    I intend to keep my SpamCop email address as my permanent address for the foreseeable future. For now I am very happy with my POPFile setup not only because it sorts spam for me, but also because it can sort other types of mail as well, such as newsgroups or work related mail. If and when JT upgrades to SpamAssassin 3, I may try it's filtering over my POPFile setup or perhaps even in conjunction, not sure yet.

    *** Note that POPFile's IMAP support has only been around for 3 or 4 months, so my above setup has not been in place for that long. Prior to this I still always relied on IMAP, but utilized all blacklists and a SpamAssassin setting of '3' I believe.

  13. Error: Cannot log into IMAP mailserver as <insert email address here>

    I am seeing the exact same error message, but allow me to clarify where I (and I assume others) am seeing this. I am *only* seeing this when trying to access my held mail on the VER page.

    I can log into webmail fine.

    I can view my held mail in Horde/IMP fine.

    I can report mail from my held mail folder in Horde/IMP fine.

    I am seeing the error message when I attempt to view my held mail while at the SpamCop website (not Horde/IMP.) This error is 100% reproducible for me at this time.


  14. POPFile is bayesian based filtering software.

    A couple of sugestions for Cosmido:

    1) Take POPFile out of the picture (temporarily disable) and determine if you can forward spam To SpamCop successfully.

    2) Are you using the extension for POPFile that enables reporting to SpamCop (created by Jonathan Michaelson)? If so, then you should check out this thread here in the POPFile extensions forum: http://sourceforge.net/forum/forum.php?thr...forum_id=226152


  15. Actually, I'll admit that I fell asleep while going through the 334 Bug Tickets on your referenced site.

    Fantastic! That means I helped Wazoo get some shut eye.

    I was loooking through the bug list too and did not turn up anything. When choosing to display only bugs (resolved or otherwise) for IMP I got a list of 107 items. I also checked their mailing list but found no one describing similar issues to what SpamCop mail users are seeing.

    I also noted that the current cvs is being referred to as IMP version 4, so apparently, after the recent v 3.2.4 the next release will be 4.

    For reference, bugs can be checked here:

    http://bugs.horde.org/search.php (pick queue IMP)

    Also, the mailing list for IMP is archived by

    gmane and can be accessed by http or nntp:



    I might submit a message to the IMP mailing list asking if anyone else has seen these symptoms ever.


  16. Most helpful Steve. From looking at Horde's IMP page it appears that IMP 3.2.2 was released on 2003-08-27, so this would likely be the version JT has setup currently. I also see that IMP 3.2.4 was released earlier this month.

    My motive was not to ask for an upgrade, but for other reasons. Since IMP is open source software it is relevant to consider that some bugs SpamCop users experience when using IMP have nothing to do with SpamCop at all. It may be useful to peruse the known bugs of a given version of IMP to determine if a particular issue that a SpamCop user experiences is widespread. Not sure if they use bugzilla or what, have not looked... But this is mainly why I was wondering about the version.

    Maybe a quick check of IMP's bugs might shed some light on the problem that people are seeing when you suddenly end up up in a different folder than the one you expected. It is possible that the source of the problem could be IMP and then the errors seen when reporting spam are only result of the IMP bug.

    All threads I am aware of on this issue below for reference:





    Wazoo -- Any chance of merging some of these threads as you think appropriate? It would certainly help track the problem over time.


  17. Does anyone know what version of Horde/Imp SpamCop is running? Or better yet, if the version is displayed anywhere once a SpamCop mail user logs in?

    I have not seen the version number displayed anywhere, but it would be useful to display it so as newer versions are released and JT upgrades then users that care will know.