Most recent edit on 2008-06-27 08:00:26 by CedderS [add hypothetical explanation of discrepancies]

Additions:
You may get an alert and summary, but nothing to the abuse address because one or more of the following apply:

• the Reporter has deselected the abuse address, but still submitted it as a report
• you have chosen not to accept munged reports
• it's gone to a trap address and no registered user has reported it

You may get user Report, but no summary because:

• it is a Spamvertized URL Report, an an informational message about a website referenced in reported spam
• because SpamCop has identified the email as backscatter,
• a relay the email passed through has already had an alert (?).
  
  

  ---

  
  Edited on 2008-06-27 07:32:11 by CedderS [bullets]
  
  Additions:
  The next step (http://www.spamcop.net/fom-serve/cache/343.html∞) is to ensure that http://www.abuse.net/∞ has current and specific contact information for the domain name that corresponds to the reverse DNS for the IP addresses involved - it very often hasn't. Sometimes postmaster@ is sufficient, but you can always email the abuse.net automaton to correct outdated information.
• If there is a hole in a user script, disable it, stop your MTA temporarily, and put all messages in the queue that contain a spam string on hold. Patch the script (if you have time) and inform your user.
• If it looks like one of your users is simply sending something to everyone in their address book, forward the report to them explaining that someone thought their email was spam, and if it's the first time and in good faith gently suggest it's not the done thing and remind them of the terms and conditions of the email service that you provide. You might also want to suggest a mailing list function that includes an unsubscribe link at the bottom of every email.
• If it is a mailing list, and it does include an unsubscribe link, and VERP is enabled, I regard it as acceptable to unsubscribe that user and notify the list owner.
• Some emails appear just to have been reported by accident, and these are the hardest to know what to do with. Some sysadmins may ignore them, but you may want to contact the Reporter via SpamCop (by replying to the message) to get further details.
• If the email is an opt-in confirmation (e.g. someone is spamming a list-subscribe address with a spoofed sender), there really is very little that can be done.
• Ensure all your users have strong passwords for SMTP authentication
• For PHP/Apache:
  • try to turn off allow_url_fopen as default, although unfortunately some software requires it. As of 2007-8, there are a great number of searches for holes in user PHP scripts which can be induced to include a foreign script. This can also often be prevented by setting register_globals off.
  • set an open_basedir for each site, and/or use PHP4/5's safe_mode. Safe mode means email sent via PHP will have the envelope sender of the the web server, minimising backscatter, and allowing you to collect bounces. You can also patch PHP ( http://www.lancs.ac.uk/~steveb/patches/php-mail-header-patch/∞ ) to show what script is being abused.
  • install mod_security2 and the default rule sets
• keep a keen eye on netstat connections using outgoing SMTP
  
  Deletions:
  The next step (http://www.spamcop.net/fom-serve/cache/343.html∞) is to ensure that http://www.abuse.net/∞ has current and specific contact information for the domain name that corresponds to the reverse DNS for the IP addresses involved. Sometimes postmaster@ is sufficient, but you can always email the abuse.net automaton to correct outdated information. If there is a hole in a user script, disable it, stop your MTA temporarily, and put all messages in the queue that contain a spam string on hold. Patch the script (if you have time) and inform your user.
  If it looks like one of your users is simply sending something to everyone in their address book, forward the report to them explaining that someone thought their email was spam, and if it's the first time and in good faith gently suggest it's not the done thing and remind them of the terms and conditions of the email service that you provide. You might also want to suggest a mailing list function that includes an unsubscribe link at the bottom of every email.
  If it is a mailing list, and it does include an unsubscribe link, and VERP is enabled, I regard it as acceptable to unsubscribe that user and notify the list owner.
  Some emails appear just to have been reported by accident, and these are the hardest to know what to do with. Some sysadmins may ignore them, but you may want to contact the Reporter via SpamCop (by replying to the message) to get further details.
  If the email is an opt-in confirmation (e.g. someone is spamming a list-subscribe address with a spoofed sender), there really is very little that can be done.
  Ensure all your users have strong passwords for SMTP authentication
  For PHP/Apache:
  
  try to turn off allow_url_fopen as default, although unfortunately some software requires it. As of 2007-8, there are a great number of searches for holes in user PHP scripts which can be induced to include a foreign script. This can also often be prevented by setting register_globals off.
  set an open_basedir for each site, and/or use PHP4/5's safe_mode. Safe mode means email sent via PHP will have the envelope sender of the the web server, minimising backscatter, and allowing you to collect bounces. You can also patch PHP ( http://www.lancs.ac.uk/~steveb/patches/php-mail-header-patch/∞ ) to show what script is being abused.
  install mod_security2 and the default rule sets
  
  keep a keen eye on netstat connections using outgoing SMTP
  
  

  ---

  
  Edited on 2008-06-27 07:29:32 by CedderS [link on backscatter]
  
  Additions:
  See also http://www.spamcop.net/fom-serve/cache/329.html∞

Most importantly, secondary mail servers should check validity of recipient and sender addresses before passing onto a primary. Ideally, mail should not be accepted into your network if it might later be rejected, so the database/files used by the secondary should be in reasonably close synchronisation (say, hourly) with the primary.

Deletions:
Most importantly, secondary mail servers should check validity of recipient and sender addresses before passing onto a primary. Ideally, mail should not be accepted into your network if it might later be rejected, so the database/files used by the secondary should be in reasonably close synchronisation (say, hourly) with the primary.

---

Oldest known version of this page was edited on 2008-06-27 07:20:41 by CedderS [started (lengthy)]

Tips For System Administrators

This page is intended for people who may be recipients of SpamCop reports and who want to make best use of them to avoid sending spam and backscatter. It is written particularly from the perspective of small ISPs, web hosts or institutions, who, like 95% of system administrators - or at least 95% of those who aren't worked to death, want to do the responsible thing while keeping their system clean. As such, you will have systems that are patched up-to-date, with all non-essential services firewalled or turned off, subscribe to security alerts for your operating system, and have a clear Acceptable Use Policy that you are prepared to remind users/customers of as necessary. It is then unlikely that you will have a server listed in the SpamCop blocklist, but you want to avoid the possibility and in any case reduce abuse reports.

There is an expanded and updated version of the FAQ on the forums that you should read first: http://forum.spamcop.net/forums/index.php?showtopic=2238#HFAD∞

There is some general information to compare with the abuse desks of larger ISPs at http://www.maawg.org/about/publishedDocuments/Abuse_Desk_Common_Practices.pdf∞

Getting as much information as possible from SpamCop

Before you start, you will want to check that SpamCop has the correct reporting addresses for the IP block that you control. By 'control', I mean not just servers that you have root/administrator access to, but also any dedicated servers that you host for someone else and have allocated an IP address to: you can then forward reports to the responsible admins and encourage them to sign up to reporting services themselves. To check the abuse addresses are correct, you can use a SpamCop reporting account to submit the IP address in question, or simply query http://www.spamcop.net/sc?track=1.2.3.4∞.

If this is incorrect, check the whois information: in a shell, type "whois 1.2.3.4", or look up via http://www.arin.net/whois/index.html∞ (you may need to do a subsequent RIPE, APNIC etc web query to get the final details for the IP block.) If the whois information is incorrect, you will need to register the details with your regional NIC, or get your upstream provider to do so on your behalf.

The next step (http://www.spamcop.net/fom-serve/cache/343.html∞) is to ensure that http://www.abuse.net/∞ has current and specific contact information for the domain name that corresponds to the reverse DNS for the IP addresses involved. Sometimes postmaster@ is sufficient, but you can always email the abuse.net automaton to correct outdated information.

You should then start to receive any abuse reports from SpamCop within a day or two.

If you are keen to avoid being blocked by specific ISPs, some have their own feedback loops, such as AOL: http://postmaster.info.aol.com/cgi-bin/fbl.pl∞ However, the AOL reporting mechanism is generally used by less technically-literate users than SpamCop, who just click on "This is spam", without checking whether it is in fact a mailing list that they have deliberately subscribed to. Many users have understandably decided that it is counter-productive to follow unsubscribe links or reply to the sender, and instead submit an unexpected email to a reporting service that preserves their anonymity by munging their address. The usual way around this, to obtain the complainant address so that they can be unsubscribed, is to use VERP
(Variable envelope return path) for example in Mailman (VERP_PERSONALIZED_DELIVERIES = 1) or some other way of individualising SMTP id (in Postfix set smtp_destination_concurrency_limit=1 ). The downside of VERP is that it can increase the size of your mail queue by a factor of ten under adverse conditions.

Of the types of report listed at http://forum.spamcop.net/forums/index.php?showtopic=4540∞ , it is most likely you will just receive "spam Source Reports" and "Spamvertized URL Reports". You certainly should not receive anything about open relays, unless something is seriously wrong in your MTA configuration. The source reports include in the Subject line either the IP address involved (for source reports) or the URL involved (for URL reports). If the same email refers to your network both as the host of the URL and the source of the email, only one email will be sent to you (with the URL in the subject line), although it should be treated primarily as a source report. See http://www.spamcop.net/fom-serve/cache/89.html∞

Of the "Spam Source Reports", some may be marked in the body text "(IP address) appears to be sending unsolicited bounces".

Summary reports

You can also subscribe to alerts and summary reports for your IP ranges, which can be helpful confirming large-scale abuse of your system. You can either do this using your abuse-desk address, or create a separate "ISP Account" where these should be sent, which is distinct from a SpamCop reporting account, but is basically unrelated to email sent to your abuse-desk address although the web interface is similar. See ISPAccount.

In most cases you will want to use the abuse-desk address for most things, add all your network addresses to the routes, and ensure you get all reports, and hourly summaries. If a particular innocent URL on your system is repeatedly referenced in UBE, you can mark that issue as resolved, or elect not to receive "spamvertized URL reports".

Summary reports include columns for "Trap", "User", "Mole" and "Simp". Usually you will only see an individual report for the incidents listed under "User". Only spam Source Reports contribute to the statistical total for "User", not URL Reports. From experience, it seems "unsolicited bounces" also do not contribute to the "User" total, and sometimes when alerts also include substantial numbers of "trap" hits (during an intrusion), summaries you receive may also not reflect "Spam Source Reports".

What use are the summaries of trap hits?

The spamtrap addresses have only been used to seed websites, and so have been harvested by webbots. Any value of Trap >0 therefore means that some spam-related activity is occurring, although it may just be that the spammer is spoofing the spamtrap address and something on your system is generating backscatter. Because more information about spamtrap hits isn't usually given (even an exact time) except on writing to the SpamCop deputies, it can be frustratingly hard to pin this source down. Again, see ISPAccount

Q: if user reports can be anonymised and sent to an abuse address, why can't the same happen with trapped email?

In my experience, a value of Trap=1 on an hourly summary probably indicates backscatter, while a value of Trap>1 indicates an intrusion. If you have an alert about high CPU or MTA queues, and on examining the queue find a lot of emails written in a foreign language and are unable to immediately determine if they are a legitimate mailout or an abuse, then receiving a SpamCop alert and statistical report may confirm the latter.

Summary reports for your own network can also be used to judge the effectiveness of an abuse desk and of anti-spam measures.

Reducing unsolicited bounces (backscatter)

Most importantly, secondary mail servers should check validity of recipient and sender addresses before passing onto a primary. Ideally, mail should not be accepted into your network if it might later be rejected, so the database/files used by the secondary should be in reasonably close synchronisation (say, hourly) with the primary.

Any anti-virus scanner should be configured not to send reports to the sender email address. Even more importantly with anti-spam. It may be feasible to do pre-queue processing for viruses so that you can reject during an SMTP session, but typically pre-queue processing for spam is too time-consuming, particularly if you have one server handling both incoming and outgoing mail. Splitting these functions ("split mx") can help in many ways. Given that it is hard to reject spam during the SMTP session (rejecting based on BLs or SPF/DKIM failure may be acceptable in some institutions and very small providers but not usually when handling customers with diverse needs), the main way of preventing spam reaching both the user and any auto-responder, is to send suspected spam to a quarantine. This of course will reduce the amount of backscatter although borderline spam (e.g. SpamAssassin hits 5-10) still needs to go to the user.

Mailman 2.1.x contains auto-responders for each list, including -request, -unsubscribe and -subscribe addresses. This is intended be removed as a default option in Mailman 2.2 and 3.0.

Many users demand vacation or "out-of-office" responses. A possible solution to prevent vacation or mailing-list autoresponders is to do preliminary checking in the MTA or proxy (such as amavis), allow email through to mailboxes, but prevent spammy strings from triggering the autoresponder. For a standard *nix .forward file for email that has been through SpamAssassin, you could do something like:

\user, "| tee $HOME/lastmsg | grep -c \"SPF_FAIL\\|SPF_SOFTFAIL\\|X-Spam-Level: \\*\\*\\*\\*\\|DCC_CHECK\\|HTTP_IP\\|FROM_APACHE\\||List-Unsubscribe:\\|yahoo\\.co\\.jp>\" >/dev/null || cat $HOME/lastmsg | /usr/bin/vacation user"

Dealing with source and URL reports

Inevitably you are going to have to triage reports you receive. SpamCop does not recommend particular methods to deal with spam sources. These are just suggestions from one recipient of SpamCop reports:

General security tips

Obviously a lot of information is duplicated elsewhere on the web.

---

CategorySpamCop
CategoryPagesUnderConstruction

---