Not getting all mail to abuse@ ?

[Cedders]

We're an ISP/host with a range of 63 IPs. Our correct abuse[at] addresses come up for this range, and we do get occasional SpamCop user reports (3 so far this year, last on 25 May). However, we have also subscribed other postmaster addresses to get alerts and summary reports for the same range. Sometimes we get these ("[spamCop] Alert"), but there is no corresponding email to abuse[at] With just the IP address specified in the alert, but no message id (or X-PHP-scri_pt header, which is useful for detecting intrusions into client Apache sites), it is harder to act on these.

I've checked the server logs this last time it's happened but see no attempt to connect from SpamCop or to abuse. Occasionally the opposite happens too: We see [spamCop (123.123.123.123) id..] but if there is any summary for that IP, it comes 5-7 days later, and shows a '1' under 'Trap', and a '0' under 'User'.

Is there any reason this might happen? Mostly when we get the summary but not the full report, it is listed as "Trap: 1" and refers to one of our IPs which we actually sub-let to a sister network - however, we are still the abuse contact for this IP address if I feed it into SpamCop. Should we get full reports for "Trap"? Also when we do get the summaries, there is a link to mark the issue as closed, but shouldn't that be done by the abuse[at] address? As I understand it, anyone can get 3rd party summaries for any range.

I may be missing something obvious here. Any explanation appreciated. Thanks to SpamCop for a great reporting tool, although sometimes I wish it were easier to find info on the help pages.

[StevenUnderwood]

If I understand properly, things are working as expected.

You will NOT receive any reports for spamtrap reports since there is no defense for these and providing full details would expose the trap address. They have never been used except to seed websites. You will only receive summaries for spamtrap reports. You may be able to get a little more information by contacting deputies[at]admin.spamcop.net address.

You do not provide an IP address or range to check. You can check where reports should be going by entering your IP into: http://www.spamcop.net/sc?track=1.2.3.4

[Wazoo]

I may be missing something obvious here. Any explanation appreciated. Thanks to SpamCop for a great reporting tool, although sometimes I wish it were easier to find info on the help pages.

Missing for starters ... all the hints, pointers, suggestions, clues on Hpw to ask a question .... No way for anyone here to try to look anything up when all you provide is the bogus 123.123.123.123 IP Address.

I can't even guess at what you might be calling "help" pages. There's the Original/Official FAQ, there's the single-page-access-expanded version of that FAQ found 'here' .. there's a Dictionary, a Glossary, a Wiki .... thousands of existing Topics / Discussion started and contributed to by other users .... any "search" done on the word 'trap' would surely have returned coutless loads of hits that actually referenced 'spam trap' and / or 'spamtrap' ... which is a term 'defined' in several places. There's a 'search' box on the Official/Original FAQ page, there's an internal search tool and a Google search tool provided 'here' .... searchable places include the Original FAQ, this Forum, the newsgroup archives, the active (last 90 days) of the 'current' newsgroups .... kind of hard to guess at what problems you are having with which resource with the "wish it were easier" phrase tossed out.

No one receives notification of a SpamCop.net spamtrap hit .... however, that incoming spam does count as a heavily weighted variable used in the calculation used in getting an IP Address listed/delisted in the SpamCopDNSBL, which is a fact included in zillions of places.

Your apparent "interested third party" setting is an option that can be safely ignored/checked/unchecked by the 'reporting' SpamCop.net user. The described action of "subscribing other postmaster addresses" simply doesn't sound right, so I'm not rally sure what you actually mean .... probably just pointing again to the start of this paragraph.

There are numerous types of Reports sent out, you really didn't mention which ones you are actually receiving (other than your mention of summary reports) ... I believe your "5-7 days" thing is part of 'your' subscription process in which 'you' selected the frequency of these summary reports, apparently 'you' went with weekly .....

[Miss Betsy]

Since I am not an ISP nor a spamcop employee, I can only guess at what you are asking. Hopefully, someone who knows more will come along soon.

I think, from previous discussions, that what you have signed up with spamcop for are a count of reports from users and spamtraps. The reason you don't get a full report to the abuse address is that the users do not want to send a report for fear of letting spammers have their address and spamtraps do not send reports since they are email addresses that have never sent email so any email they get is unsolicited.

The only good I can see for the count report is that if you have an infected computer hitting spamtraps, you might get warned before the IP address is listed on spamcop, but that's no good if the report comes 5-7 days later. And like you, I don't know why you should be able to 'close' an issue since the summary count has so little information, you couldn't be sure that you had 'fixed' it. I doubt that 3rd parties who get reports have the ability to 'close' an issue. Only the owners of the IP block would be able to 'close' it. The people who sublet from you could get 3rd party reports if you had not been passing the reports along to them. Occasionally, there are owners who don't pass spamcop reports to subletters and the subletters want to receive them so that they can fix any problem.

But no, you will never get a full report from 'trap'.

Miss Betsy

[Cedders]

Thanks for the replies. Sorry if I'm not explaining things properly.

Steven answered it, really. So 'Trap' reports don't send the full message to the abuse address in order to protect the honeypot addresses? That seems slightly odd to me at first because if user reports can be anonymised, why can't trapped email? On reflection, I can see why they are protected - otherwise spammers who gain access access to an abuse[at] address might more easily be able to develop an exclusion list, without any benefit for an end user who would be all too happy to be excluded. I can't think why any servers in our range would be sending to trap addresses unless there's an intrusion or exploit (most likely) or abuse by a client who has somehow added harvested addresses. OK, maybe it's an autoresponder sending through the server in question. I guess we rely on users (of Spamcop, feedback loops, or just directly) to report an offending email at the same time, but they don't, which is what confused me as recipient of the summaries.

Yes, I tested the IP address and the abuse addresses are correct. I selected hourly summaries - there's no option for weekly. And http://www.spamcop.net/mcgi?action=listroutes lists the ISP account (abuse2[at]) as "Third party interested in daily aggregate summary reports", but doesn't come up as an optional reporting address when you submit a spam. That's fine and as expected.

It still leaves the less important puzzle of why user reports aren't always reflected in the summaries. If people are reporting a spam than incidentally includes a link to an innocent website, then it makes sense too that this is not listed in a summary of spam sources. http://www.spamcop.net/fom-serve/cache/89.html says reports about email sources are more serious than about website addresses, but if both happen in the same message, the website is given in the subject line (could this be a bug?). However, I think that that report, although it doesn't mention the IP the email originated from, probably counts as "User: 1". Sometimes it looks like user reports (where we do get most of the email and can see what is going on) get listed as "Trap", but it's possible both are happening at the same time.

OK, we're getting there... I'm looking at all email received to the abuse and abuse2 addresses from reports.spamcop.net or admin.spamcop.net; and that leaves one other occasion where there is a report about mail from an IP, but no corresponding summary at all. This one is described as "appears to be sending unsolicited bounces", so maybe those are regarded as not serious enough to add to the summary? (It is an out of office message originating on a client's MSExchange server - we exclude SPF softfails and suspected spam from the vacation system on our own server.) Then there's one other last year when there was a genuine intrusion, one user report for the originating IP, but no summary or alert for that IP, although there was a helpful trap report for the relay it passed through, also on our network.

By 'help pages' I meant the pages shown when you click 'Help' at the top right of the screen. Although there's lots of interesting information under 'Help for abuse-desks and administrators', I couldn't find a manual page or other centralised documentation about how to make the most out of the reports. Yes, there is a proliferation of wiki pages and other sources, and even a Usenet server - it's the multitude of sources that's daunting. I'm not saying the information isn't there somewhere, but I have spent over an hour looking and if someone could provide a link to where the relationship between summaries and individual reports, and the corresponding relationship between abuse addresses and ISP accounts, is described, I'd be grateful.

I didn't give the exact IP address partly because while it's in our rack space it's used by a different organisation, to whom we forward the reports for action. (Perhaps they should have a direct abuse address for those IPs, but they're mostly run by volunteers.) Also, I don't like to admit publicly that sites we host might have had professional intruders, however quickly I'd like to think we detect, stop, and patch against it. SpamCop admins could deduce the full details from my email address for this forum, although there's no point unless there is access to log files. I'm just trying to work out what's happening using these forums - if something still seems not right (it's hard to say "not as documented", because I haven't found the documentation), then I'll contact the deputies via email (I hope logs go back a year).

Finally, we never get anything listed under 'Simp', which I'd presume we'd get if a message were reported using, for example, spamassassin -R. What sort of weighting do these get?

So to summarise - there are SpamCop ISP accounts, which are available to anyone to get hourly/daily summary reports and alerts, and which are distinct from the ability of an abuse desk to declare an issue resolved (quite rightly, but I still don't understand what the 'Close Issues' action on the drop down menu in the ISP account "Control Centre" does). There is the implication at http://www.spamcop.net/fom-serve/cache/266.html that there is a link between ISP account and abuse address, although that isn't necessarily so, e.g. where we have abuse[at] and abuse2[at] - in order to be able to set preferences and alerts for an abuse desk, a password for the public abuse address needs to be requested at http://www.spamcop.net/denied.shtml.

Where there is an alert and summary to the ISP account, but nothing to the abuse[at] address, it could be because the reporter has chosen not to send the report, the abuse address had unwisely chosen not to accept munged reports, or that it's gone to a trap address and no registered user has reported it. Trap alerts are still useful to confirm compromise of a server. Where there is a report sent to the abuse desk, but nothing to the ISP account, it could be because it is a more informational message about a website referenced in the reported message, or because SpamCop has identified it as backscatter, or possibly because a relay it passes through has already had an alert (?). In short, alerts and reports arriving at similar times should not be expected necessarily to correlate to the same incident.

[Miss Betsy]

spamtraps are email addresses that never send email so they can't send reports. That's the main reason, I believe. A secondary reason is to protect the email address as you have guessed.

The reason there is little 'help' here is because the users who get summaries have never contributed any 'help' on how they use the summaries or what they have learned about the summaries from using the summaries. This is a 'user' forum.

Perhaps the summaries do not include actual reports made because, presumably, the report has gone to the abuse desk. In other words, you either get a report or you get a summary of mole reports and spamtrap hits. As you have noted, often spamtrap hits are 'misdirected bounces' or automatic replies to spam such as out of office replies. And that is why server admins find them useful, perhaps. Misdirected bounces are considered just as serious as actual spam for the spam blocklist - for some people, they can almost be like a DOS attack.

If the IP address that is having most of the problems is run by volunteers, then I would suggest that you give them more help when you forward a report. My experience with volunteers is that many seem to leave their common sense at the door or are doing things way beyond their expertise and need explicit directions on what to do to avoid reports.

You probably need to understand something about how the spamcop blocklist works. No reports about spamvertized websites (innocent or not) contribute to the spamcop blocklist and none come from spamtraps (because spamtraps don't send reports). However, any hits on spamtraps have a much heavier weight for listing the source IP address (Another reason, perhaps, why summaries perhaps are useful to server admins). There is a complicated algorithym that controls listing and delisting.

About revealing the IP address: Spamcop blocklist is almost entirely automatic (deputies can correct mistakes, but that is the only human intervention). Therefore, there is no need to be afraid that spamcop would be more aware of problems on that IP. Although this forum does have spammers visit it, I think they generally rely on other methods to discover a 'weak' IP address.

The 'close issue' generally means that you have found the reason for the spamcop reports and have fixed it. It is not to be used lightly since, if you haven't found the problem and fixed it, and the problem continues, then it cannot be used again.

Another reason for the discrepancy between full reports from reporters and the summary list is that, perhaps, no reporter received that particular spam or misdirected bounce while the spamtrap did or vice versa.

Thank you for being conscientious about keeping your servers and the IP addresses under your control free of outgoing spam. If there were more server admins like you, the spam problem would be much easier to control.

Miss Betsy

[Telarin]

I would think that if it is listed in the summaries, and is not a trap address, there should be a report to the abuse[at] address to go along with it. I would suggest sending an email to deputies[at]admin.spamcop.net, as they would be able to offer you a lot more insight into what might prevent reports from being sent. However, make sure you are very specific when emailing them, and include your IP range, and any other pertinent information so they can look up the data as quickly as possible.

[StevenUnderwood]

I would think that if it is listed in the summaries, and is not a trap address, there should be a report to the abuse[at] address to go along with it.

The mole status, which still exists though not very useful also adds to the summary reports but send no reports, is still available as far as I know.

[Telarin]

Ahh yes, I forgot about mole reports. But those don't contribute as heavily to the BL do they?

[StevenUnderwood]

Ahh yes, I forgot about mole reports. But those don't contribute as heavily to the BL do they?

I believe they are not counted at all, but that information is a bit old and may have changed. Weight=0 is the phrase sitting in the brain.

[Wazoo]

So 'Trap' reports don't send the full message to the abuse address in order to protect the honeypot addresses?

Spamtrap addresses have nothing to do with the technical term 'honeypot'

That seems slightly odd to me at first because if user reports can be anonymised, why can't trapped email? On reflection, I can see why they are protected - otherwise spammers who gain access access to an abuse[at] address might more easily be able to develop an exclusion list, without any benefit for an end user who would be all too happy to be excluded.

Assumptions made, most of them wrong. SpamCop spamtrap addresses are not 'abuse[at]' addresses. On the other hand, it has been seen that some spammers will in fact include SpamCop.net spamtrap addresses in their spew from a previously 'clean' (but now compromised) ISP/Host e-mail server, primarily to provide a 'nice' introduction to the results of a SpamCopDNSBL listing for the previously unaware Admin of this new-found spam source, basically trying to spread the 'bad reputation' of SpamCop.net.

I can't think why any servers in our range would be sending to trap addresses unless there's an intrusion or exploit (most likely) or abuse by a client who has somehow added harvested addresses. OK, maybe it's an autoresponder sending through the server in question.

Any and all of what you said, in addition to my previous comment.

It still leaves the less important puzzle of why user reports aren't always reflected in the summaries.

Again, no way for anyone here to attempt any research based on the lack of specific data provided in your query. IP Addresses involved, URLs involved, Tracking URLs, etc.

If people are reporting a spam than incidentally includes a link to an innocent website, then it makes sense too that this is not listed in a summary of spam sources. .... but if both happen in the same message, the website is given in the subject line (could this be a bug?). However, I think that that report, although it doesn't mention the IP the email originated from, probably counts as "User: 1". Sometimes it looks like user reports (where we do get most of the email and can see what is going on) get listed as "Trap", but it's possible both are happening at the same time.

A lot of mixing and matching going on in that paragraph. For example, you signed up for a Summary Report for an IP Address. That Summary Report would contain no data about a spamvertised URL .. two totally different items.

By 'help pages' I meant the pages shown when you click 'Help' at the top right of the screen. Although there's lots of interesting information under 'Help for abuse-desks and administrators', I couldn't find a manual page or other centralised documentation about how to make the most out of the reports. Yes, there is a proliferation of wiki pages and other sources, and even a Usenet server - it's the multitude of sources that's daunting. I'm not saying the information isn't there somewhere, but I have spent over an hour looking and if someone could provide a link to where the relationship between summaries and individual reports, and the corresponding relationship between abuse addresses and ISP accounts, is described, I'd be grateful.

Your reference to 'help' and Subject headings indicate that you are talking about the Official/Original FAQ as found at www.spamcop.net. It was the years of complaints like yours, complaints about it not being updated, expanded, fixed, etc. that led to my Saturday morning hack which then led to the expanded singel-page-access version found 'here' links at the top of this very page. I have also installed and tried numerous other attempts at building a 'live' FAQ, but the one still standing is the Wiki.

SpamCopReportTypes hasn't really been updated / fully worked up.

SpamCop Report Types seems to have developed a nasty issue with displaying HTML for some reason .... work needed to fix that

Addition to SpamCop Report Types, add ISP Statistical Reports .. it's been so long, I'm betting that Miss Betsy forgot that she did this page <g>

I didn't give the exact IP address partly because while it's in our rack space it's used by a different organisation, to whom we forward the reports for action. (Perhaps they should have a direct abuse address for those IPs, but they're mostly run by volunteers.) ... SpamCop admins could deduce the full details from my email address for this forum, although there's no point unless there is access to log files.

Without IP Addresses, specific URLs, there isn't much that the volunteer users can do as far as any research needed to answer specific questions. There isn't a court/jury involved here (well, for the most part <g>) // this is a support Forum. //

Finally, we never get anything listed under 'Simp', which I'd presume we'd get if a message were reported using, for example, spamassassin -R. What sort of weighting do these get?

My instructions from the Deputies .... I can't talk to this point. I will note that Miss Betsy's FAQ entry referenced above includes a statement about 'Simple' ....

So to summarise - there are SpamCop ISP accounts, which are available to anyone to get hourly/daily summary reports and alerts, and which are distinct from the ability of an abuse desk to declare an issue resolved (quite rightly, but I still don't understand what the 'Close Issues' action on the drop down menu in the ISP account "Control Centre" does). There is the implication at http://www.spamcop.net/fom-serve/cache/266.html that there is a link between ISP account and abuse address, although that isn't necessarily so, e.g. where we have abuse[at] and abuse2[at] - in order to be able to set preferences and alerts for an abuse desk, a password for the public abuse address needs to be requested at http://www.spamcop.net/denied.shtml.

Actually, a few of your comments here leave me somewhat confused, especially the URL you toss out about a password. On the other hand, I will state that the one person with the needed inside knowledge that has read this Topic/Discussion has chosen not to make a public post to answer your query. I can't answer as to why this is so. As you stated previously, perhaps you need to contact him directly.

Noting also that the only persons with access to your Registration details here are exactly two, and one of those folks spends very little time here. Neither person on that short list has access to the Parsing & Reporting System. Please see Section 8 - SpamCop's System & Active Staff User Guide

Where there is an alert and summary to the ISP account, but nothing to the abuse[at] address, it could be because the reporter has chosen not to send the report, the abuse address had unwisely chosen not to accept munged reports, or that it's gone to a trap address and no registered user has reported it. Trap alerts are still useful to confirm compromise of a server. Where there is a report sent to the abuse desk, but nothing to the ISP account, it could be because it is a more informational message about a website referenced in the reported message, or because SpamCop has identified it as backscatter, or possibly because a relay it passes through has already had an alert (?). In short, alerts and reports arriving at similar times should not be expected necessarily to correlate to the same incident.

Again, it's hard for 'us' to look anything up with no specific data provided.

I believe they are not counted at all, but that information is a bit old and may have changed. Weight=0 is the phrase sitting in the brain.

Correct. Mole Reports only add to the statistics as seen via the Summary Reports and as additional data for the Deputies when a decision has to be made.

[Merlyn]

a lot of lengthy discussion going on but if you want assistance then just post the ip(s) in question.

[Miss Betsy]

My instructions from the Deputies .... I can't talk to this point. I will note that Miss Betsy's FAQ entry referenced above includes a statement about 'Simple' ....

And you are right - I don't remember anything about it! It looks to me as though I copied (and maybe edited to look like a FAQ) a post by Steven Underwood. And I do have a vague memory of thinking that it would be a good idea to keep the research done by Steven for future reference, but that I really didn't understand the discussion since it was among server admins and used a lot of jargon that I am not sure of.

from the FAQ "Simp: Simple reports - messages submitted by unregistered users." Obviously, when I posted that FAQ, I didn't really read it. Because how in the world can an 'unregistered user' make a report? I don't wonder that the OP doesn't get many of them. It must be a deal spamcop has with some server admins (or maybe even one of those 'used' email addresses that have been abandoned to spam that people are always offering as spamtraps - those email accounts could send reports, but really couldn't register). Don't know how the OP gets 'simple' as connected to spamassassin, but it looks to me as though he is kind of like me - I probably could, with a little initial help, learn how to administer an email server without having any obvious spamming holes and know enough to know where to look for obvious problems and how to stop them, but I still wouldn't remember the correct technical term for a lot of things and my guesses would be not even close to 'educated guesses' - just like the guess above about simple reports.

And it does say that one of the categories is reports from 'Users' - I am betting that the OP is thinking that all reports should show up in the summary and only source reports do.

And I am also betting that the OP won't post an IP, but will email the deputies (or has done so) and we will never know anything more. I suspect, since we have so few questions posted, that either few server admins use the summaries or that they are so experienced that, if they have a problem, they contact the deputies without coming to the forum (probably have the deputies address in their address book already). Another wild guess!!

Miss Betsy

[Cedders]

from the FAQ "Simp: Simple reports - messages submitted by unregistered users." Obviously, when I posted that FAQ, I didn't really read it. Because how in the world can an 'unregistered user' make a report? I don't wonder that the OP doesn't get many of them. It must be a deal spamcop has with some server admins (or maybe even one of those 'used' email addresses that have been abandoned to spam that people are always offering as spamtraps - those email accounts could send reports, but really couldn't register).

Another sysadmin I know says he has such an arrangement.

<content elided .... as stated above, guidance from Deputies is that subject matter is not for public consumption>

but it looks to me as though he is kind of like me - I probably could, with a little initial help, learn how to administer an email server without having any obvious spamming holes and know enough to know where to look for obvious problems and how to stop them, but I still wouldn't remember the correct technical term for a lot of things and my guesses would be not even close to 'educated guesses' - just like the guess above about simple reports.

Thanks - yes, that's partly my situation. But also, when I get trap reports, I want myself or the relevant server administrator to do something about them, as in the best case it indicates a source of backscatter that might be able to be stopped if identified, and in the worst case, a compromised server. The problem is that in response to a summary with no content, and just the IP of a host that acts maybe as a web host for a hundred domains, and as a mail server, all one can do as a sysadmin is check mail logs and netstat for the previous hour, and a set of general-purpose system monitoring tools. If it's an intrusion, or an inappropriate use of (say) Mailman (or more likely, webbots submitting random addresses to a subscription forms and an issue with confirmation by the mailing-list software) it'll probably be taken care of anyway, but the sysadmin won't even know for sure that's the cause of the trap report.

And it does say that one of the categories is reports from 'Users' - I am betting that the OP is thinking that all reports should show up in the summary and only source reports do.

That was an additional potential source of confusion, but as I said in my second post in this thread, referenced websites (sometimes just quoting a site to give the message some authority), don't or shouldn't of course show up in the summary report since they are no reflection of the spam source. So it doesn't explain all the discrepancies unfortunately.

I believed that "spamvertized" sites were contributed by SpamCop to URIBL / SURBL, but maybe that's wrong.

And I am also betting that the OP won't post an IP, but will email the deputies (or has done so) and we will never know anything more.

Indeed I intend to email the deputies shortly with the IPs concerned and full details. I'd also like to contribute a wiki page aimed at sysadmins, saying in my experience how best to respond to trap reports. (If the wiki admins enable this account, I can do so in due course.)

I suspect, since we have so few questions posted, that either few server admins use the summaries or that they are so experienced that, if they have a problem, they contact the deputies without coming to the forum (probably have the deputies address in their address book already). Another wild guess!!

I think you're right, and it's mostly the latter. I just thought I'd try out my questions here before submitting a private enquiry. But I doubt I'm the only recipient of reports/summaries/alerts with these questions.

[Wazoo]

Appearances seem to be that I'm wasting my time trying to answer your questions, provide data an links to data.

But also, when I get trap reports,

It has been stated repeatedly, by different folks ... there is no such thing as you getting "spamtrap reports" .. once again, spamtrap hits do not generate reports.

as in the best case it indicates a source of backscatter that might be able to be stopped if identified, and in the worst case, a compromised server.

SpamCop.net spamtrap addresses are available for scraping. The point being that your 'out-of-office' type of backscatter should not happen, as the spamtraps do not generate any e-mail. In your compromised-server scenario, the gist would be that the spammer is intentionally adding a SpamCop.net spamtrap address t to the list of recipients .. but again, I addressed this in my last post. Much more typical is the use of a dirty list of 100% opted-in participants, for sure!!

The problem is that in response to a summary with no content, and just the IP of a host that acts maybe as a web host for a hundred domains,

Again, it seems that all my words in my previous post were simply ignored. No links followed to dig up the background data.

That was an additional potential source of confusion, but as I said in my second post in this thread, referenced websites (sometimes just quoting a site to give the message some authority), don't or shouldn't of course show up in the summary report since they are no reflection of the spam source. So it doesn't explain all the discrepancies unfortunately.

To which I repeatedly state ... my previous post ran across this ground, pointed out some facts and concepts, offered up links, on and on. And the killer statement remains, without IP Addresses, URLs involved, no one here can do any research. This has been pointed out by several folks that have been willing to try to give you some help.

I believed that "spamvertized" sites were contributed by SpamCop to URIBL / SURBL, but maybe that's wrong.

This is a FAQ entry in the single-page-access-expanded version of the SpamCop FAQ found 'here' ... but as you didn't seem to follow my last set of links, I'm not going to look up and post the URL here.

Indeed I intend to email the deputies shortly with the IPs concerned and full details. I'd also like to contribute a wiki page aimed at sysadmins, saying in my experience how best to respond to trap reports. (If the wiki admins enable this account, I can do so in due course.)

OK, so it seems this Topic/Discussion wants to be moved to the Lounge area, as the quest for "help" doesn't seem to be working. A lot of folks trying, but without specific data provided, generalities and hypotheticals just won't cut it.

As much as we'd like the assist in developing Wiki pages/content ... the continued use of the wrong terminolgy here, the appearance that zero research was done prior to making this last statement, seemingly to have totally ignored my last post .... just doesn't leave me feeling that I want to get all excited and bypass the procedures long set in place for Wiki Registration.

I think you're right, and it's mostly the latter. I just thought I'd try out my questions here before submitting a private enquiry. But I doubt I'm the only recipient of reports/summaries/alerts with these questions.

Generally, when a SysAdmin arrives to talk about Summary Reports, it's usually to state that they are pretty much useless except as a warning indicator that "somethig is happening" .... numerous previous Topics/Discussion within this Forum structure.

[StevenUnderwood]

I believed that "spamvertized" sites were contributed by SpamCop to URIBL / SURBL, but maybe that's wrong.

Mostly symantecs but the SURBL just pulls the public data provided by SpamCop to feed its list. It has never been stated whether SpamCop officailly supports this use of it's data, but apparently has not blocked access to it. It has been stated multiple times by SpamCop employees that finding Spamvertized links is not a high priority of the development time.

[Wazoo]

Moving this waste of yime to the Lounge area, as it seems apparent that the Topic Starter did not actually want any specific help .. which is not the goal of the SpamCop Reporting Help Forum section. Basically going to have to tak ethis as a really poor rant of some sort.

[Miss Betsy]

No, I don't think that it was a rant. I think the OP really did want to know how to use the summary reports. As he pointed out, the only real use is as an alert to look at your logs and see what might be causing a problem. He wanted to correlate them with actual spamcop reports which is not possible because they come from different sources - reports from users (not part of summary) vs a count of non-contributing to the bl /non-sending to the abuse desk reports from users and spam trap hits contributing to the blocklist, but sending no reports.

Also, if the OP wanted to contribute a FAQ, he could contribute it to the forum - the rules don't have to be bent for the Wiki.

However, I don't think he will get any different advice or information, even with URLs, from the deputies. I don't think they have time to divine what he really means and he isn't techie enough to ask a succinct question that they will answer.

Miss Betsy

[Cedders]

Appearances seem to be that I'm wasting my time trying to answer your questions, provide data an links to data.

It has been stated repeatedly, by different folks ... there is no such thing as you getting "spamtrap reports" .. once again, spamtrap hits do not generate reports.

I meant the messages headed "[spamCop] summary report" that are described in your link http://forum.spamcop.net/forums/index.php?showtopic=5619 and look something like:

Start/Length Trap User Mole Simp Comments

Jun 17 08h/0 1 0 0 0

I should have said "spamtrap summaries" for clarity.

SpamCop.net spamtrap addresses are available for scraping. The point being that your 'out-of-office' type of backscatter should not happen, as the spamtraps do not generate any e-mail.

But spammers fake From headers and envelope senders from addresses in their database. So 'out-of-office' (or mailman automaton etc.) backscatter to trap addresses will happen unless the server admins take conscious steps to prevent it.

In your compromised-server scenario, the gist would be that the spammer is intentionally adding a SpamCop.net spamtrap address t to the list of recipients .. but again, I addressed this in my last post.

Yes, you seemed to be saying the same thing earlier, but I think you are mistaken. The spammers cannot of course tell the difference between harvested addresses of active mailboxes belonging to people to annoy, and harvested addresses of spamtraps. So it isn't intentional when a spammer either sends to a spamtrap address or fakes the spamtrap address when sending elsewhere, causing backscatter to be sent to trap addresses.

Much more typical is the use of a dirty list of 100% opted-in participants, for sure!!

Typical of the internet as a whole perhaps, but atypical in the non-commercial arena in which I work.

Again, it seems that all my words in my previous post were simply ignored. No links followed to dig up the background data.

I followed the links to the other forum postings, but couldn't find answers to my questions there except at http://forum.spamcop.net/forums/index.php?showtopic=7818 . Steven Underwood has already posted that information here. It would be useful to have it in the main FAQ at http://www.spamcop.net/fom-serve/cache/75.html, because the amended version on the forum looks sufficiently similar that I initially assumed it was identical.

To which I repeatedly state ... my previous post ran across this ground, pointed out some facts and concepts, offered up links, on and on. And the killer statement remains, without IP Addresses, URLs involved, no one here can do any research. This has been pointed out by several folks that have been willing to try to give you some help.

Well the link above states "Only the SpamCop Deputies have access to detailed information about e-mail hitting spam traps", which implies to me that posting IP addresses here won't help. There is no URL involved, since any information about the content of the spamtrap hit isn't included in the summary report.

Generally, when a SysAdmin arrives to talk about Summary Reports, it's usually to state that they are pretty much useless except as a warning indicator that "somethig is happening" .... numerous previous Topics/Discussion within this Forum structure.

Well, just regard me as one of those, but with a few more detailed questions and a bit more verbose . That Summary Reports aren't intended as anything other than a warning indicator (useful in differentiating legitimate user mailouts from intrusions) and aren't helpful in tracing backscatter or user abuse is important information that I can't find within either the main or amended version of the SpamCop FAQ.

Cheers

Cedders

[Miss Betsy]

I followed the links to the other forum postings, but couldn't find answers to my questions there except at http://forum.spamcop.net/forums/index.php?showtopic=7818 . Steven Underwood has already posted that information here. It would be useful to have it in the main FAQ at http://www.spamcop.net/fom-serve/cache/75.html, because the amended version on the forum looks sufficiently similar that I initially assumed it was identical.

Unfortunately, there is nothing that can be done about the main FAQ.

Do you have a suggestion on how one could find it better /here/?

Miss Betsy

[Wazoo]

I should have said "spamtrap summaries" for clarity.

Clarity for whom???? There is no such thing. There is a "Summary Report" .. which includes a count position for spamtrap hit data, but that is just one variable.

But spammers fake From headers and envelope senders from addresses in their database. So 'out-of-office' (or mailman automaton etc.) backscatter to trap addresses will happen unless the server admins take conscious steps to prevent it.

Yes, you seemed to be saying the same thing earlier, but I think you are mistaken. The spammers cannot of course tell the difference between harvested addresses of active mailboxes belonging to people to annoy, and harvested addresses of spamtraps. So it isn't intentional when a spammer either sends to a spamtrap address or fakes the spamtrap address when sending elsewhere, causing backscatter to be sent to trap addresses.

History and experience are that some spammes have in fact done what I described.

I followed the links to the other forum postings, but couldn't find answers to my questions there except at http://forum.spamcop.net/forums/index.php?showtopic=7818 . Steven Underwood has already posted that information here. It would be useful to have it in the main FAQ at http://www.spamcop.net/fom-serve/cache/75.html, because the amended version on the forum looks sufficiently similar that I initially assumed it was identical.

Some of the question you seemed to have asked do not have answers as your underlying basic facts are not correct. Noting for instance that the lead-in introduction to the single-page-access-expanded version of the SpamCop FAQ as found 'here' desribes and defines why the Official/Original FAQ situation is what les to my starting and offering all the other alternatives forms, versions, resources 'here' ... on the other hand, you are welcome to pick the tattered flag and start your own crusade to try to get that thng updated, corrected, and expanded.

Well the link above states "Only the SpamCop Deputies have access to detailed information about e-mail hitting spam traps", which implies to me that posting IP addresses here won't help. There is no URL involved, since any information about the content of the spamtrap hit isn't included in the summary report.

You are the one that has repeatedly brought up Domain names/URLs in your 'questions' .... You are the one that mentioned receiving a spam Report here, but no confirming number in the Summary Repprt. This is where the offer of assistance was made by several folks, doing the research on what (public / subscriber) information exists to try to answer what it appeared that you were asking about.

Failing that additional data request action, reading your replies, looking at the timeline, it seemed apparent that you had moved on to doing the "e-mail direct to Deputies" thing .. which is why I moved this to the Lounge area. Yet, you make no mention of any on-going dialog with the Deputies on any of this.

Well, just regard me as one of those, but with a few more detailed questions and a bit more verbose . That Summary Reports aren't intended as anything other than a warning indicator (useful in differentiating legitimate user mailouts from intrusions) and aren't helpful in tracing backscatter or user abuse is important information that I can't find within either the main or amended version of the SpamCop FAQ.

Noting that you didn't include the Wiki in this last comment. What about ISP Account or How can I get SpamCop reports about my network?

[Wazoo]

No, I don't think that it was a rant.

Lack of a better word .. attempting to describe the amount of activity generated in this Topic with no way to resolve or even define anything. As noted, even the official SpamCop.net person has chosen not to get involved publically for whatever reason.

I think the OP really did want to know how to use the summary reports. As he pointed out, the only real use is as an alert to look at your logs and see what might be causing a problem.

As the complaints go, as the previous dialog has gone, these are "Summary" Reports. Cedders has eventually stated that he/she has in fact seen the FAQ entries on the "types" of Reports sent out ... though still leaves some confusion floatng around about just what types of Reports have actually been seen/received, sliding around the mentions of 'sites' in one sentence, IP Addresses in another sentence, then heading back to Summary Reports ..???? The last post in which "Domains have not been mentioned" really gets my head spinning, when compared to the various descriptions of hosted sites, spam that included URLs, etc.

He wanted to correlate them with actual spamcop reports which is not possible because they come from different sources - reports from users (not part of summary) vs a count of non-contributing to the bl /non-sending to the abuse desk reports from users and spam trap hits contributing to the blocklist, but sending no reports.

I think I know what you meant, but .... data for a Summary Report comes from the same place. Counts for the various types of activity are offered up as a "Summary" .... Looking for "Details" requires other work, access, and effort.

Also, if the OP wanted to contribute a FAQ, he could contribute it to the forum - the rules don't have to be bent for the Wiki.

Actually, this would be much preferred. The continued misuse of words, definitions, descriptions would be much more easily handled here in the Forum in addition to (probably) having more folks taking a look at it before comitting it to the Wiki. Noting that historically, getting a Wiki account that then allowed posting and editing there has been basically limited to folks that contributed to the Forum. "Contributed" basically defined as helping other folks, answering questions, and the like. Personally, I don't see that scenario being met yet.

However, I don't think he will get any different advice or information, even with URLs, from the deputies. I don't think they have time to divine what he really means and he isn't techie enough to ask a succinct question that they will answer.

Will not argue your points at all, but will suggest that it's the range of IP Addresses, Domains, URLs, etc. involved that I beleive will hinder a fantastic outpouring of a Deputy response. The "I'm the Contact" here, an "Interested Third-Party" there, an "I'm the Admin" on this one, "I'm a friend of the site owner" on that one .... just way too much confusion involved. Just my opinion, which is also based on the lack of specific details being provided.

[Miss Betsy]

I don't have time to research it, but my impression is that the Summary does /not/ include data from reports sent to abuse desks by reporters. The Summary reports include data from reports by users who do not want to send reports to abuse desks - these reports are not counted against the scbl - and from spam traps that do not send reports at all, but do contribute to the scbl. The reason, of course, is that a report sent by the reporter has all the data needed to track down the problem and that the Summary contains reports where little or no data is available to the server admin, but only points to a possible problem which may not have been reported by a reporter. It is a way of answering those server admins who complained that their IP address got on the blocklist without them receiving a report.

IMHO, 'volunteers' should be as competent as paid staff in whatever they are attempting to do. I don't hold it as an excuse that they are 'volunteers' and don't know any better.

Miss Betsy

[Wazoo]

I don't have time to research it, but my impression is that the Summary does /not/ include data from reports sent to abuse desks by reporters. The Summary reports include data from reports by users who do not want to send reports to abuse desks - these reports are not counted against the scbl - and from spam traps that do not send reports at all, but do contribute to the scbl.

Even as you defined the contents in the page referenced prior that's exiting under your name; the contents of a Summary Report include;

Start/Length Trap User Mole Simp Comments

The "User" column being the SpamCop Reporters that are not Mole Reporters.

The difference being the Reporting system attempting to send out a Report on the User generated Reports. Failure to receive them might be as silly as an incorrect or inappropriate e-mail address, an e-mail address that had in turn been "turned off" due to bouncing, recipient action by selecting the "Do Not Send any more Reports" or Deputy action to stop sending reports to that address.

[Cedders]

Unfortunately, there is nothing that can be done about the main FAQ.

That is indeed a shame, as it's the place I would imagine most sysadmins would look for information about reports. There are links to the main site from the reports to get more report details.

Do you have a suggestion on how one could find it better /here/?

Well, people will most likely look for help from the main site (in which case they see the old FAQ), or a web search, in which case they might find information here. Therefore if you have a discussion that turns up something useful, you may want as many search terms and synonyms mentioned on the one page as are relevant.

In particular, can I suggest a new forum specifically titled "For abuse-desks and system administrators"? The perspective of a Reporter is different from the perspective of a sysadmin, which is different again from the perspective of a SpamCop expert. I only fall into the first two of those, and perhaps should also point out that most sysadmins probably don't have as much time as me.

I see a sysadmin got a similar response to a query here: http://forum.spamcop.net/forums/index.php?showtopic=9392 (although in that case the numbers seemed to indicate actual customer abuse rather than occasional backscatter). Again, there is an issue of confidentiality about an IP address when you want to uphold the reputation of the organisation you work for and its customers. Maybe it shouldn't be so, and sysadmins should let it all hang out, but it's generally responsible to get consent from several other people.

IMHO I don't think a forum is the best medium for a FAQ, compared to static HTML, since it lists stuff chronologically rather than logically. I gather you intend to migrate help content to the Wiki, which I think would make it more accessible. Having the two apparently identical indexes 'SpamCop Discussion' and 'Discussions & Observations', and two different pinned links to the same FAQ, also confused me.

In any case, I have made additions to http://forum.spamcop.net/scwik/ISPAccount since I found it misleading (it was about receiving summaries and alerts, not reports themselves), and added a lengthy page at http://forum.spamcop.net/scwik/TipsForSystemAdministrators giving some context that is not necessarily specific to SpamCop, but likely to be relevant to sysadmins, and summarising what I have found out in the last few days.

Clarity for whom???? There is no such thing. There is a "Summary Report" .. which includes a count position for spamtrap hit data, but that is just one variable.

Clarity for other people on this forum including visitors and to distinguish a summary report from a Report. What I was calling a "trap report" was a summary report listing one trap hit with all other variables as zero. I hope that is clear. As you may have gathered, these can be confusing for recipients since there is no additional information about these spamtrap hits that means you can take action.

Some of the question you seemed to have asked do not have answers as your underlying basic facts are not correct. Noting for instance that the lead-in introduction to the single-page-access-expanded version of the SpamCop FAQ as found 'here' desribes and defines why the Official/Original FAQ situation is what les to my starting and offering all the other alternatives forms, versions, resources 'here' ... on the other hand, you are welcome to pick the tattered flag and start your own crusade to try to get that thng updated, corrected, and expanded.

You are the one that has repeatedly brought up Domain names/URLs in your 'questions' .... You are the one that mentioned receiving a spam Report here, but no confirming number in the Summary Repprt. This is where the offer of assistance was made by several folks, doing the research on what (public / subscriber) information exists to try to answer what it appeared that you were asking about.

Well, I guess I have access to public/subscriber information myself - ideally what I wanted was someone with access to the SpamCop source code, or detailed documentation for it. I didn't bring up domains or URLs; I thought you did.

Failing that additional data request action, reading your replies, looking at the timeline, it seemed apparent that you had moved on to doing the "e-mail direct to Deputies" thing .. which is why I moved this to the Lounge area. Yet, you make no mention of any on-going dialog with the Deputies on any of this.

I emailed them at the deputies[at]admin address, but haven't heard anything back yet. Maybe they're taking a break. I included the reports from SpamCop in question as attachments. I know succinctness is appreciated, but I did have four separate questions.

Noting that you didn't include the Wiki in this last comment. What about ISP Account or How can I get SpamCop reports about my network?

It gave instructions for getting alerts and summary reports only. Third-party status wasn't the issue, and I think it is likely to be far less common than simple queries you might get about interpreting and acting on summaries, hence my additions.

Lack of a better word .. attempting to describe the amount of activity generated in this Topic with no way to resolve or even define anything. As noted, even the official SpamCop.net person has chosen not to get involved publically for whatever reason.

As I say above, I think the communication problem stems from coming from different perspectives. I imagine other people who have staffed an abuse email have a similar perspective on SpamCop to me.

As the complaints go, as the previous dialog has gone, these are "Summary" Reports. Cedders has eventually stated that he/she has in fact seen the FAQ entries on the "types" of Reports sent out ... though still leaves some confusion floatng around about just what types of Reports have actually been seen/received,

(1) User spam source reports and (2) "spamvertized URL reports" to the main abuse address; and (3) alerts and (4) summary reports to a secondary address (abuse2[at]). I assumed that would have been clear from my first post, at least to anyone else who was similarly subscribed. Types (1) and (2) are described in http://forum.spamcop.net/forums/index.php?showtopic=4540 ; type (4) is described at the bottom of http://forum.spamcop.net/forums/index.php?showtopic=5619. Whether (3) and (4) fall under "Report" with a capital "R" in the SpamCop definition, I doubt, even if (3) is described as a "summary report" in the subject line. I am trying to use the same terminology as the SpamCop system itself uses.

sliding around the mentions of 'sites' in one sentence, IP Addresses in another sentence, then heading back to Summary Reports ..????

If I referred to 'site' previously I would probably have meant a virtual host on a web server, usually with one domain, sometimes with additional aliases, owned by one user account. There are usually multiple sites on one 'IP address' (and usually only one IP address per 'host' or 'server') and multiple IP addresses in one 'network' or 'block'. This is how I understand the terms, anyway, and am unaware of any different meaning they have in SpamCop. A summary report may summarise one or more hit.

The last post in which "Domains have not been mentioned" really gets my head spinning, when compared to the various descriptions of hosted sites, spam that included URLs, etc.

I'm sorry, I'm a bit confused now too. I don't see the post that says "Domains have not been mentioned". Someone complaining about a spam may include URLs etc. I'm not. I'm trying to make sense of SpamCop reports (specifically summaries and alerts) and get more information about them. The spam source reports contain the body of the Reported email, but summary reports do not.

Keep fighting spam!

I don't have time to research it, but my impression is that the Summary does /not/ include data from reports sent to abuse desks by reporters.

I'm almost certain it does increment the number under 'User', but the actual spam source report containing the message details (i.e. what you need to handle the complaint) is sent separately. The summaries come from summaries[at]admin.spamcop.net, while the other reports come from userid [at]reports.spamcop.net

The Summary reports include data from reports by users who do not want to send reports to abuse desks - these reports are not counted against the scbl - and from spam traps that do not send reports at all, but do contribute to the scbl.

Well, I've just tried submitting a spam (from outside my network) and unticking the report address - this just gives an empty SpamCop page. I'd hope it doesn't contribute to the BL (except as Mole) because it would be confusing for sysadmins, and a little unfair and open to abuse.

The reason, of course, is that a report sent by the reporter has all the data needed to track down the problem and that the Summary contains reports where little or no data is available to the server admin, but only points to a possible problem which may not have been reported by a reporter. It is a way of answering those server admins who complained that their IP address got on the blocklist without them receiving a report.

That seems logical. The main problem is when there is no data available - as in the case of the trap hits.