spamcup, account prefs, and defaults

[stevenworr]

I looked, I researched, I read, I'm exhausted, and I have no idea if this is the right place to ask, but I really tried the best I could.

I am running spamcup to confirm all of the spam that I report. I have taken great pains to make sure that the messages I report are exactly what I want reported and that the innocent will not be blamed.

In my account settings, I have set Public standard report recipients to point to a coldrain.net address run by knujon. Spamcop lets me set the field and the instructions say

If you wish others to receive a copy of every spam you submit, enter the email address here. Please do not send to any address which is not receptive to receiving untargetted spam reports. Note this will create only one copy for each spam, even if there are multiple reports per spam.

When the spam is reported and I get the request to confirm, I go there and that box for the PSRR field is filled in with the correct address but it is not checked by default.

There's also a field called 3rd party report default which is set to Send by default.

Since I'm not using the web interface, knujon never gets their copy of the message.

Is this a bug? Is it a feature? Should I have seen the answer somewhere else?

TIA

[rconner]

I'm confused when you say you don't use the web interface. Normally, if you forward your spams to SpamCop, you then have to go to the web interface to inspect them and rack them up for reporting.

When you see your list of spams on the web interface, the menu at the top lets you choose "report-and-trash" (I'm paraphrasing), which is the time-saving option but does not let you report web links or use the PSRR. If you pick "queue for reporting," then you have to review each spam, but you can then use the PSRR.

I too have KnujOn as a PSRR, and it is also unchecked by default for me. That's OK, because I don't always use it.

I hope this helps.

-- rick

[Miss Betsy]

I can see why you chose 'Suggested tools..." but I think probably you would get better responses in the Spamcop Reporting Forum or the Email Forum.

And, I think I remember a discussion about why the 'Public Standard Recipients' is not set to default. Basically, because not all spam would necessarily go to knujon or wherever (i.e. 419 scams rarely include a website. At least I can't ever remember seeing one and some people don't report to knujon, but do like to send phishes to a special investigative address and of course, not all spam are phishes.)

Since I don't have a spamcop email account, I don't remember what the '3rd party' option is for or why it would be defaulted to send while the other is not. The only 3rd parties I can think of are the '3rd Party interested in receiving spamcop reports' that sometimes show up on spamcop reports. Some people don't want to send to '3rd parties' because they are not sure about the motives of the '3rd party' - I believe some tracked them down and discovered that some 3rd parties were spammers. It is difficult to get spamcop to send reports to 3rd parties because of that. Some people, however, do not want spammers knowing they are reporters, either out of fear of reprisals or because they don't want to be listwashed (so they can continue to keep these sources on the list). These people objected to having to uncheck those boxes so maybe this was the resolution - you get to choose in preferences whether reports to 3rd parties are checked or unchecked. We will see if my guess is correct.

And don't feel bad about not finding information about either subject. I am no good with searching either - which is why I haven't provided any links. Well, I decided to try and came up with this one (second item in search) Public Standard Report I also found information on 3rd party, but it was very old and I didn't try to see if anything was newer. Maybe you didn't use the search box at the top of the forum? The search box in the menu (below the Reporting Server Status graph) is very, very clunky.

Miss Betsy

[stevenworr]

"I'm confused when you say you don't use the web interface. Normally, if you forward your spams to SpamCop, you then have to go to the web interface to inspect them and rack them up for reporting."

I'm using spamcup which is a commandline tool for reporting. spamcup has an option to approve all reports without confirmation. That's what I'm using.

Also, What I'm trying to get is whether the PSRR should be checked by default BECAUSE I checked 3rd Party report default

[rconner]

I'm using spamcup which is a commandline tool for reporting. spamcup has an option to approve all reports without confirmation. That's what I'm using.

Well, live and learn. I'd never heard of Spamcup before, I thought your fingers were slipping on the keys!

Also, What I'm trying to get is whether the PSRR should be checked by default BECAUSE I checked 3rd Party report default

PSRRs are different from "3rd parties" as SpamCop defines the term, so I don't think that the 3rd party checkbox has any effect on the PSRRs. In short, the 3rd parties are people that SpamCop picks, while the PSRRs are people that YOU pick. There was a dust-up some years ago over one of these 3rd parties, which precipitated the rather odd checkbox option you tried to use. You can find some history here on the forums.

-- rick

[Wazoo]

I looked, I researched, I read, I'm exhausted, and I have no idea if this is the right place to ask, but I really tried the best I could.

Not sure I'd agree with your choice.

Suggested Tools and Applications

A Forum for pointing to those neat fixes and solutions that we've all been looking for. Free is great, open-source even better. Noting issues with Spyware/Adware would also be appreciated.

Yet, what you've offered up is a third-party tool that doesn't do what you think it should, describe it as a means to circumvent your responsibilities when choosing to use the SpamCop.net Parsing & Reporting System. Definitely doesn't fit into my thoughts on what I'd set this Forum section up for.

I am running spamcup to confirm all of the spam that I report. I have taken great pains to make sure that the messages I report are exactly what I want reported and that the innocent will not be blamed.

I don't really understancd all of your words here (and in follow-on posts.) What I really seem to be reading is that you use this other tool to handle your submittals, but you don't actually look at the results. Not good at all.

In my account settings, I have set Public standard report recipients to point to a coldrain.net address run by knujon.

......

When the spam is reported and I get the request to confirm, I go there and that box for the PSRR field is filled in with the correct address but it is not checked by default.

This doesn't seem to match your description of the spamcup tool ?????

There's also a field called 3rd party report default which is set to Send by default.

Since I'm not using the web interface, knujon never gets their copy of the message.

Is this a bug? Is it a feature? Should I have seen the answer somewhere else?

As I'm not following your example description, expecially the part about the "not reviewing the results before sending out Reports" .... all I'm going to say is try the 'search' tool using //coldrain// as your query. There's been a whole passle of Topics/Discussions, some fairly recently, about folks using them, questions about the chekboxes not being defaulted, and a few examples of how other folks are handling 'simultaneous' submittals .. on and on .. take a look at http://forum.spamcop.net/forums/index.php?showtopic=10229 for example, say about the third page (default view)

Still trying to decide whether to just move this Topic or actually merge it into an existing Topic/Discussion. It is a Reporting issue, but .... your descrption of the thrid-party tool in use doesn't really want me to leave any hint of suggesting that it's of any 'good' value to those not technically knowledgeable about all of the possible impacts. No, this is not the first time spamcup has come up within this Forum.

[rconner]

I found Spamcup at Sourceforge. The bad news appears at the top of the page:

As of 2009-03-11 00:00, this project is no longer under active development.

The worse news is found in the bug tracker where people have frequently complained about incompatibilities (the most recent of the tickets is still open after more than two years). Many a slip 'twixt Cup and Cop, it seems. The Spamcup tool would be very vulnerable to obsolescence whenever SpamCop decides to change the HTML it generates (it wouldn't even have to change the parser's mode of operation, just the HTML that gets sent to the browser). No doubt it could be patched to keep it up to date, but why?

Looks as though Cup simply fetches web pages from Cop, fills in the necessary form data, and then returns the forms. Same thing a human would do, except that the web browser is bypassed. As Wazoo notes, other important stuff that a human user needs to see might also be getting bypassed.

-- rick

[Miss Betsy]

Yes, I also thought it was just a typo!

It doesn't sound as though his problem is really to do with spamcup, though. I don't see the purpose of spamcup anyway. How can it tell whether the email is spam or not? Why doesn't he use the forward as attachment method if he doesn't like filling in web forms? Maybe because he is using one of those where you have to put the headers in one box and the content in another?

Well, if anyone is checking out spamcup, they will now know something about it.

Also, What I'm trying to get is whether the PSRR should be checked by default BECAUSE I checked 3rd Party report default

The answer to your question is NO.

If you want an explanation of why the answer is NO, then re-read the previous posts or search in the topmost box for Public Standard Reports and 3rd Party. Lots of information turned up for me.

Miss Betsy

[stevenworr]

I've gotten some feedback from you all and I think the suggestion for this material being sent to the SpamCop Reporting Help forum is probably the best idea.

I would like to add a couple of points that seem to have confused people (for which I apologise):

* I have been using spamcop for many years.

* I do an excellent job of only reporting only that which is really spam.

* I used to report only those messages that got though my spam filters (i.e, the false negatives).

* I am running my own mail server.

* I use spamcup (spelled with a U) as a commandline tool to confirm all of the spam that I send to spamcop.

* I have spamcup set to not give me the option to look at the report before confirming. When I start the program it runs to completion and confirms everything.

* I just switched my server settings over so that whereas I used to only send those few messages that made it through the filter (about < a half dozen per day) I now send all messages that are caught. This means that spamcop is now getting between 800-1200 messages per day from my server.

* Confirming all of them manually is not feasible and I do an excellent job of not reporting people's messages who should not be reported. There has not been a slip-up in at least two years (that I know of). If spamcop offered an option to not require post confirmation, I would take it and I would take it with confidence at how well my system works.

* The part that I'd like to get fixed is to get the knujon address to be cc'd. I can't do that from the spamcup interface because even though the entry is in the form, it's not by default checked. Spamcup only confirms the whole default report.

* Spamcup has not been actively supported for a few years but it's a small perl scri_pt and it seems to work quite well. And the price was right.

I'll certainly monitor this thread, but I'll start it over on the Spamcop Reporting Help forum.

Thanks to all.

[stevenworr]

I started this discussion on the Suggested Tools and Applications forum and I took it as far as I could, but now I'm here and I'm hoping to do better. My situation is slightly more complex than the average person so please bear with me.

Moderator Edit: Posted into;

SpamCop Discussion > Discussions & Observations > How to use .... Instructions, Tutorials

SpamCop Reporting

How to Instructions Only - Problems and issues belong in the other specific Forum sections

As this is a follow-on to a current Topic/Discussion, adding much more information, I am going to "Merge" this 'new' Topic into that existing one (which had also been moved to the pb[Reporting Help Fotum section.

I run my own mail server in my own personal domain on a linux box on my cable modem. I'm running sendmail and spamassassin. In the past, the few messages that got through (i.e., the false negatives) would be sent to spamcop. All of the mail I send to spamcop is automatically confirmed by a commandline tool called spamcup. It works great, there are no problems with it and that's a good thing because it has not been supported for a few years. Basically, it's a small perl scri_pt. When I run spamcup, I set it to not ask for confirmation. It just blindly confirms everything. I take pride in doing an excellent job of not reporting anything that doesn't deserve it. NO MISTAKES.

I used to send approximately a half dozen per day to spamcup. I only sent those that had gotten through spamassassin (i.e., the false negatives.) I run a few local mailinglists on my server using Majordomo2 and all email is properly equipped with identification and unsub instructions. The total number of subscribers is ~1500 with 10-20 messages per day on average. Besides the half dozen that I reported to spamcop, the total number of spam messages per day is around 800 - 1200. I do not give accounts to people but I wanted to mention this so you have a flavor what what I'm doing.

I recently made a tweak to my server to cause all rejected messages to be sent to spamcop. I also set up a job to run every 15 minutes to run spamcup and confirm everything. So nothing sits on spamcop for longer than that. I do this simply because I want to contribute. Right now a reasonable estimate for the total amount of spam in the world is running between 60%-80% of the total.

Recently, I attended the Annual MIT spam Conference. This year, I met the people who created the Knujon.org operation. They are also looking for spam feeds. Their thrust is to work with legislators and DA's to stimulate prosecutions. They are really doing good work.

I saw in my spamcop account that I can set Public standard report recipients (PSRR) to point to a coldrain.net address that was issued to me by knujon. I set that field in my account to the coldrain address. I also saw a field in my account setting called 3rd party report default which I set to Send by default.

Everything is working well. The only problem I have is that the confirmation form returned to me by spamcop does not set the checkbox in the PSRR field. It is unchecked. So, when I run spamcup, all the spam is properly confirmed but knujon is not getting their copy.

I thank everyone for reading this far. Now I can ask my question:

Is this a bug that the PSRR field is not checked even though I have set the 3rd party report default set to Send by default?

If this is not a bug, is there some other setting I can use to cause a copy to be sent there?

PLEASE let me know if you need more information. Like I said, this is not a vanilla scenario and I'd like to help with the data I have available.

TIA

[Miss Betsy]

It is not a bug. People get various kinds of spam. As I said before, 419 scams rarely, if ever, contain a website URL. There is no purpose to send them to knujon. Some people like to report 419 scams somewhere else. So the box would be there for them to report 419 scams. It would depend on the type of spam that they received which box they would check.

If you want to report your spam to knujon every single time, you will have to find another way to do it.

Something that I think you have missed is that the spamcop parser is a tool in anti-spam fighting. You can use it to find the source IP address or website URL without reports; you can use it to send reports to the source IP address without having to address the email yourself. It also sends reports to spamvertised website owners. However, the creator of spamcop did not think that notifying spamvertised websites was as important as reporting - <i>and blocking<\i> source IP addresses. He was going to discontinue reporting spamvertised sites altogether when spammers started the botnet and fast flux business, but some people objected because they use spamvertised sites to block incoming spam. I am not sure exactly how it works, but somehow the spamvertised sites that spamcop finds help them. The parser doesn't attempt to find many that are obfuscated (which people are always complaining about). If you search properly, I am sure that you will find long discussions on whether it is useful to try to shut down spamvertised websites vs blocking source IP addresses. IMHO, knujon's philosophy is not internet friendly. I much prefer Complainerator if I were going to do more than simply report spam to the source.

Again, the point is that spamcop is a tool. You are responsible for seeing that the reports go where they are supposed to go. If you want to send hundreds to knujon automatically also, then find a tool that does it automatically. Ask on the knujon help board is anyone has a tool to do it automatically.

Spamcop is not the tool to use to report anything automatically except the source IP address.

Miss Betsy

Miss Betsy

[stevenworr]

Ok. Someone is confused and it may very well be me. My understanding is that knujon wants all email spam, not just spam that contains URLs to web scams. I just went to the knujon website to confirm that and it was my understanding based on the talk we got at the MIT spam Conference.

I agree that spamcop is a tool, but I don't see that as an impediment for what I want to do. What am I missing? Everything that I report is spam and it should all be processed. No?

[Miss Betsy]

Maybe they want all spam because they know that trying to explain to people how to differentiate between what they want and what they don't want is a losing battle.

I don't have time (or inclination, frankly) to look up their mission statement, but I am fairly sure that what they do is close down spamvertised websites. They don't say, "Don't send us 419 spam" but if there is no website, what are they going to do with it?

What you are missing is that spamcop is not responsible for the reports that you send. You are. If you want to submit all your spam automatically to knujon, then ask knujon for an automatic way to do that. Spamcop is set up to be used to report source IP addresses and you can do that automatically without using a 3rd party add on tool. The parser can also do some other things as well if you want to use it that way. One of the things it doesn't do is to send spam automatically wherever you want it to go. It should be obvious why not.

Miss Betsy

[stevenworr]

I think we're getting sidetracked. Knujon actually wants all email that is spam. There is nothing from them that indicates that they want only spam that contains URLs to websites. And! Just in case they are not getting what they want they are free any time to let me know so I can make modifications.

Either way, we do agree that I am responsible for all of the reports that I send out. Further, I am very comfortable with the quality of the selection process and the accuracy of the reports that I send out.

Can we please get back to the question? It's very simple: I filled in a field called Public standard report recipients (PSRR) with the address that I want to use. Then I checked a box that says 3rd party report default. The text for that box (3rdPRD) says:

Sometimes, people other than the ISP as identified by SpamCop have an interest in seeing copies of SpamCop reports (for example, small downstream providers, or users running services on a DSL or cable-modem). SpamCop will offer you a checkbox on the web-reporting form when this occurs. Normally, this checkbox is default ON, but by changing this option, you can alter the behavior of SpamCop so that the default is OFF.

I am guessing that the checkbox is referring to reports that would go to other email addresses that could be viewed as interested parties. Clicking those extra addresses manually is not needed because this preference causes those addresses to be added to the list of addresses who get copies, and those addresses are by default set to checked.

Is there a way that I can cause the checkbox for the knujon email address I entered in the PSRR field to also be prechecked, so that when I run spamcup, the copy of the spam will also go to that address?

Thanks

[rconner]

Is there a way that I can cause the checkbox for the knujon email address I entered in the PSRR field to also be prechecked, so that when I run spamcup, the copy of the spam will also go to that address?

Not at present. It would undoubtedly require a code change to make it work the way you want, so this would perhaps be a "new feature request."

-- rick

[Miss Betsy]

3rd Party recipients must request spamcop to add them. As I have said, it is not an easy process because of the possibility of spammers requesting reports in order to listwash.

If you think that knujon wants all your spam, including those without websites, then suggest to the people you met at MIT to request spamcop to add them as a third party. Spamcop worked a deal with another company to send all spam to them. I forget the name and there was a hullaballu because some people didn't like the ethics of that company. That might be when the preference was added that the default with 3rd parties could be turned off. The FTC has asked that spamcop /not/ provide an automatic checkbox because they were overwhelmed with spamcop reports.

I don't think that there is any possibility of tweaking spamcop to submit automatically to knujon because of reasons pointed out before, though as rconner pointed out, you can make a new feature request.

Your best bet is to see if knujon would send your spam to spamcop as part of an automatic process or make arrangements with spamcop to be a 3rd party.

Strange, but you don't seem to be using the spamcop blocklist to filter your incoming mail. The main reason for contributing to spamcop is to take advantage of its blocklist. There are a few people who do report for altruistic reasons, but most of them do so because they do not run servers and cannot use the scbl and do not want to use spamcop email for some reason.

IMHO, the only way to slow the spam problem down is to persuade end users to demand that all spam be rejected at the server level. Since most spam is now being sent by compromised computers and botnets, it is unlikely that real mail will be rejected. If it is, then the *sender* knows and can take measures to stop it or to use a more responsible email service. Unlike knujon, I agree with spamcop that blocking the sending source of spam is the best method. I also agree with the Complainterator approach of sending reports to registrars, but that is too complicated for me to do with confidence. I do not like the approach of shutting down websites because once you allow the closing of websites, someone can always close them simply because of prejudice.

The bottom line is that some people are making money from the bandwidth used by spammers. Filter engineering has improved so much that even long time compromised email addresses get very little spam in the end user inbox. Filter engineers are also making money. Picking up litter and recycling helps our environment, but is a drop in the bucket compared to what would happen if major industries did the equivalent. The same is true of spam. Individuals can make a contribution, but if major industries like Comcast were to clean up their act and not allow compromised computers, it would make a major difference. (Actually, I don't know whether they have done so or not - it took them so long to act if they did, that I have absolutely no respect for them.)

Good luck with getting knujon to cooperate with you (and that's meant sincerely, I do wish you luck in being able to contribute to the anti-spam fight - it certainly can use more than one approach).

Miss Betsy

[Wazoo]

Some posible "out of sync" perceptions? I handled the "Suggested Tools" Topic/Discussion first .. only then hitting the 'new' Topic/Discussion in the "How to Use..." Forum section .. at which time I handled that Topic/Discussion .. which is why this Post ended up below a whole lot more 'additional' Posts. Oh well ....

* I have spamcup set to not give me the option to look at the report before confirming. When I start the program it runs to completion and confirms everything.

And this is the major objection, including being a violation of the Rules agreed to when signing up for a Reporting Account with SpamCop.net. "Submitting only spam" is but one part of those Rules, this described action circumvents the other part of those same Rules.

If spamcop offered an option to not require post confirmation, I would take it and I would take it with confidence at how well my system works.

Quick Reporting ... though noting that even this mode has the possible problem area from lack of oversight on the actual Parser results.

* The part that I'd like to get fixed is to get the knujon address to be cc'd. I can't do that from the spamcup interface because even though the entry is in the form, it's not by default checked. Spamcup only confirms the whole default report.

I previously referenced a lnk as to how other users 'solved' the problem, yet you make no eference to any of that discussion ...????

I'll certainly monitor this thread, but I'll start it over on the Spamcop Reporting Help forum.

Moving this Topic to the Reporting Help Forum section with this post.

[Wazoo]

All of the mail I send to spamcop is automatically confirmed by a commandline tool called spamcup. It works great, there are no problems with it and that's a good thing because it has not been supported for a few years. Basically, it's a small perl scri_pt. When I run spamcup, I set it to not ask for confirmation. It just blindly confirms everything. I take pride in doing an excellent job of not reporting anything that doesn't deserve it. NO MISTAKES.

...

I recently made a tweak to my server to cause all rejected messages to be sent to spamcop. I also set up a job to run every 15 minutes to run spamcup and confirm everything.

Either way, we do agree that I am responsible for all of the reports that I send out. Further, I am very comfortable with the quality of the selection process and the accuracy of the reports that I send out.

Trying to work with you here, but the above dialog really doesn't fly very well. If you are not "in control" of the outgoing Reports (because you've allowed for these various scripts to handle everything by 'blindly confirming everything') .... then there's a definite question about how you'd know about the accuracy of those outgoing Reports.

Is there a way that I can cause the checkbox for the knujon email address I entered in the PSRR field to also be prechecked, so that when I run spamcup, the copy of the spam will also go to that address?

Noting all of the objections and problems, the seemingly easy answer would appear to be a bit of tweaking of the (unsupported) scri_pt involved. But again, there are documented details offered by others as to how they 'solved' the simultaneous submittals. For that matter, as you say you've done up a cron-job, why not have it perform two outgoing feeds (or even a CC:) to both tools instead of trying to force something seen to be not-working for you?

[StevenUnderwood]

* I use spamcup (spelled with a U) as a commandline tool to confirm all of the spam that I send to spamcop.

* I have spamcup set to not give me the option to look at the report before confirming. When I start the program it runs to completion and confirms everything.

You are aware that sometimes spamcop parsing messes up and stops too early, which could mean reporting your own server or ISP multiple times? That is the primary reason behind confirming the outgoing reports, not just checking that the message really is spam, which really should be done before the send report time.

[stevenworr]

First, Thanks to Betsy, I think I may be able to fix my problem with a patch to spamcup which will click the copy to knujon for me as well as click the final send reports button.

In fact I do use the scbl as well as a number of other RBLs. The really significant change that I made to my spam filtering is that I moved all of the checks that used to be done in sendmail for the purpose of catching spam, out of sendmail into spamassassin. The problem was that when sendmail rejected a message, spamassassin never learned from it. Moving the checks now cause better management of the AWLs and Bayes tables.

Also, I'll be in touch with knujon to have them make arrangements.

And thanks to Wazoo. You provided me with good info and I'm hoping that what I say here will be useful to others:

You have legitimate concerns about how I can use spamcup to blindly confirm all submitted reports and still do the right thing. In fact, I have not had a false positive in about 2-3 years. I have a python scri_pt I wrote which runs in a cron job on a daily basis which provides a report of all mail that is rejected (and therefore sent to spamcop). I always check it to see if there's anyone in that list I know.

[turetzsr]

Hi (steven?)!

<snip>

You have legitimate concerns about how I can use spamcup to blindly confirm all submitted reports and still do the right thing. In fact, I have not had a false positive in about 2-3 years.

<snip>

...And I had the very same experience, until one day someone changed something in the e-mail environment here (not SpamCop) and I reported my own provider because I had stopped paying careful attention! Perhaps because you run you own e-mail server that danger is less (but who really knows for sure?); I nevertheless strongly urge you to look at all the reporting e-mail addresses to which SpamCop is offering to report. If there are just too many spams to do that, I'd suggest you simply send to SpamCop only those spams for which you have the time and inclination to do the review.

[stevenworr]

I really appreciate this advise. I'm shutting the flow off now. I hoped it would work but I didn't understand that there were glitches that could cause reports to be filed against people I had in my mailhosts list.

[StevenUnderwood]

I really appreciate this advise. I'm shutting the flow off now. I hoped it would work but I didn't understand that there were glitches that could cause reports to be filed against people I had in my mailhosts list.

Steve O: The reason for this forum is to keep issues public, and as such I do not converse either via PM or telephone on SpamCop issues. I hope you understand.

As Steve T stated, and I believe you now understand, changes by any of your mailhosts can cause issues with parsing. This is one of the reasons automated reporting is not encouraged here. I currently do full reporting of all my spam but have it at a level where I can do that. If I enable greylisting, I get almost no spam at all.

At work, I report only the spam I get at my account, the administrator accounts which have nver been able to send email, and a few of the past administrators which are currently not able to send email. On the administrator accounts, I unsubscribe from all valid looking newsletters as I receive them and begin reporting those as well after a year of unsubscribing. I use a template while reporting these indicating that the address has been invalid for over a year and has been unsubscribed each time it has been received.