Jump to content
Sign in to follow this  
hallikainen

Return score instead of yes/no on blocklist request?

Recommended Posts

I've been using SpamCop with sendmail for many years. As I understand it, SpamCop returns an accept/reject response based on the ratio of spam reports to queries (like the one my spamcop is sending). A lot of reports in proportion to the number of queries indicates a spammer, while a small number indicates someone is not a spammer.

What I'd like to see is for SpamCop to return the actual ratio or a score that we can use to evaluate whether to accept the connection from this IP address or not. Looking at my logs, I see a lot of spam being rejected based on the SC block list, but a lot is still getting through. I'm thinking that maybe I can make the rejection threshold a little lower so more stuff gets rejected. Right now, I have no access to that threshold. But, if a score were returned instead, each mail user could set their reject ratio.

Does this sound reasonable? Is it already running, maybe?

Thanks!

Harold

Share this post


Link to post
Share on other sites
Does this sound reasonable?

That could be useful but that is not the way the block list was designed.

To implement what you suggest one decision that would need to me made is how large a BL' would be sent. With the current setup every IP on the list is blocked. With your suggestion a compromise is needed between including all IPs with a rating above zero and some reasonable smaller size list that allows for practical use.

Another option would be a customized BL for each user.

I don't see a reasonable application. Have you tried adding other filtering criteria in addition to the SCBL?

Edited by Lking

Share this post


Link to post
Share on other sites

Once upon a time, the data presented in What is the SpamCop Blocking List (SCBL)? offered a 2% criterion. During a time when the parsing ^ Reporting system attempted to try to include IP Address blocks to try to handle the spammers gaming the system, a number of massive changes were made. Some specific details were removed from public view, real-time specific measurements ere removed from public view. The math equation went from the 2% number to a scaling result based on Reputation Points, which includes data from other sources in addition to the actual Reports.

As ststed by the powers that be, it is not recommended to use the SpamCopDNSBL in a blocking mode, rather it should be used as a scoring tool. This is actually what you seem to be asking about, wanting to do. As seen, this takes quite a bit more coding than simply turning on/selecting and usimg various BL results as a sole decision/threshold point.

This is not to ignore that this kind of data is not part of the data queries and returns from a BL.

Share this post


Link to post
Share on other sites

Thanks for the responses! I think it'd be useful for users to know how likely a particular IP address is a spam source instead of a go/no-go decision so we can set our own thresholds, but I described that before.

Thanks!

Harold

Share this post


Link to post
Share on other sites
Thanks for the responses! I think it'd be useful for users to know how likely a particular IP address is a spam source instead of a go/no-go decision so we can set our own thresholds, but I described that before.

Confused. I can only assume you have not yet looked at "the explanation" of how an IP Address gets listed/unlisted. "How likely" seems like a strange term .. if there are no complaints/spamtrap hits, there would not be a SpamCop DNSBL listing. And for the obverse, even if there are hits, the math may not work out to get listed. I simply don't understand this last remark ... spam is either seen coming from the IP Address or it's not, and if it's in sufficient quantity, the IP Address gets listed .... quantity goes down below the algebraic threshold, it gets unlisted.

As noted all over the place, the SpamCopDNSBL is a bit of a hair-trigger, almost always the first to note an issue with bad traffic (and also noted, one of the few BLs that does an 'automatic' removal after the spam stops.)

The only thing I can come up with other than the concept of "stop a spam run in progress" is that you want to make some kind of a 'game' out of a SpamCopDNSBL listing, something like ... yes, I see that there is spam coming from that IP Address, but none has made it here, so I don't want to block that Address ...???? Seems to me that this is where either the "weighting" scale (setting your own threshold) comes into play or simply make the call not to use the SpamCopDNSBL at all. There must be something that I still don't see in your request that hasn't already been addressed by the responses provided thus far.

Share this post


Link to post
Share on other sites
I see a lot of spam being rejected based on the SC block list, but a lot is still getting through.
Have you tried adding other filtering criteria in addition to the SCBL?
So you have looked at you logs and "lots" of spam are getting through.

In my experience spam filtering is like A/V no one tool will get everything. Of the 300+ spam sent to my domain yesterday one (1) made it into my READ folder and there were zero (0) false positives.

This is done with several layers of filtering.

  • My ISP filters out some. I had them turn off most of their filters for me because of their high number of false positives.
  • I use a White list to filter all the known good mail. (that is how the 1 spam got through).
  • I use a BL
  • I also use a content "word" filter to catch the topic of the day. Acai berries is a current addition to that list.

That leaves about 25-30 total in my INBOX to check. I check several times a day so there are never many decisions to make at one time. First thing in the morning is about half the load.

The one spam that got through was 1 of 9 (so far) targeted phishing spam.

Edited by Lking

Share this post


Link to post
Share on other sites

I may be wrong, but in your first post,

As I understand it, SpamCop returns an accept/reject response based on the ratio of spam reports to queries (like the one my spamcop is sending). A lot of reports in proportion to the number of queries indicates a spammer, while a small number indicates someone is not a spammer.
I understand what you said to mean that queries of the blocklist are part of the algorithm that decides what is on the scbl or not. That is not part of the equation AFAIK. How many emails normally come from that IP address does count toward being on the bl. I haven't seen a complaint in years, but it used to be biased toward smaller volume IP addresses where one or two reports would get an IP address blocklisted while larger volume IP addresses did not get listed until after a number of reports. They probably have it figured out now to be more fair.

the scbl is supposed to catch spam when the spam is being sent - of course, there is a lag between the first receiver reporting to sc and when the sending IP address accumulates enough points to be placed on the scbl when some people will get part of that spam run - which is why it is used in conjunction with spamassassin and other bls in spamcop email. For those botnets that keep rotating spamcop is not very efficient because the IP address ages off before the next spam is sent while bls that track botnets will still have them listed.

All of that is why the scbl is usually used as part of a filtering system. Also, if the IP address is a shared address, then you may not want to block it since sometimes you will get a number of false positives.

IOW, if the IP address is on the blocklist, it is sending spam. If it is not, then it may be the start of a spam run or legitimate email.

Miss Betsy

Share this post


Link to post
Share on other sites

http://www.spamcop.net/fom-serve/cache/291.html

"response code from the SpamCop server to indicate a queried IP is listed is 127.0.0.2"

not sure if that is the only response possible from spamcop (apart from the negatory one of course)

I like the idea of having the option of more blocking that would include the background static (as in radiation) spam ips.

Although with just sendmail how would this be achived if you wanted to NOT use the new response code?

FEATURE(`enhdnsbl', `bl.spamcop.net', , , `127.0.0.2')dnl

would need one of these for each response code(? wildcard/regexp possible on the last field?) but not for the response code that has a low %score (as described in original post).

Best to just have a seperate ancillary list?

A similar topic:

http://forum.spamcop.net/forums/index.php?showtopic=5892

Personally I reject solely based on 6 dnsbl with very few problems - to mitigate occassional problems encountered with various (other) blocklists I have used and currently use I employ also a whitelist in access.db - which skips the dnsbl test on those domains/addresses.

Edited by QuantumMechanic

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×