Plausibility of claim regarding false positives

[oldjack]

I use SpamCop to filter spam from my permanent address, which got about 100 spams per day. It works very well. After some tweaking of options, I got maybe 2 spams per day in my inbox and very few false postives in my held mail.

The school with which my mail provider is affiliated has installed a spam filtering system, and the number of spams reaching my SpamCop held mail has dropped from about 100 per day to about 5 per day, so it's doing some serious filtering. Under their spam filtering system, the filtered mail is discarded, and there is no way to check for false positives. I sent them an e-mail, basically expressing concern about false positives and important mail being lost. I got this reply:

"We are only discarding messages with spam scores of 98-100. Based on their experience to date, [they are] so confident that messages with scores are spam WITHOUT EXCEPTION that they are going to be using this as the default configuration for all students." (their capitalization, not mine)

Has the state of the art in spam filtering become so advanced that a system can discard 90+% of spam with no false positives?

Thanks in advance for any thoughts you might have on this.

[Farelf]

... The school with which my mail provider is affiliated has installed a spam filtering system, ...

Any idea what software they are using? Scores in one may not mean the same as the same score / number in another. You are right to have doubts - in general there can be no absolute assurance (the zero on a real-life distribution will be at infinity, if you get my drift) and, as you say, they are doing some serious filtering and it sure doesn't sound like they should be so confident in having no false positives. spam evolves, the heuristics (if any) of filtering systems can't be in front of the leading edge of that evolution. If they have no unfiltered accounts you might think to be looking for a more amenable provider. Trouble is, very many IPs seem to filter these days but not many bother to tell their subscribers.

[oldjack]

Any idea what software they are using?

It is Proofpoint Messaging Security Gateway.

[Wazoo]

It is Proofpoint Messaging Security Gateway.

http://www.proofpoint.com/products/spam.php

The large number of attributes analyzed by the MLX Anti-spam Engine ensures that messages are classified with a high degree of confidence. Most messages score very high or very low, with only 1.5% falling between 20 and 80 on a scale of 1 (an undeniably valid message) to 100 (assuredly spam).

All in what trust you want to put on the "scientists and engineers at Proofpoint's Anti-spam Laboratory" ... as then implemented by "your support staff" ...????

[Farelf]

... All in what trust you want to put on the "scientists and engineers at Proofpoint's Anti-spam Laboratory" ... as then implemented by "your support staff" ...????

Prezactly as Wazoo sayeth. If their science - http://www.proofpoint.com/products/mlx.php - is up to their sales pitch you are home and hosed. Meanwhile, back in the universe ... FANtastic, as they taught us to say at the Crunchbird School of Charm. Whatever, it appears that you can whitelist your regular correspondents and, failing that being a solution, there are "Administrator Customization" options you could negotiate with the school. Meantime, hopefully some users have experience with this product and can offer their observations which will be worth a whole lot more than my "first principles" meanderings.

[turetzsr]

<snip>

Thanks in advance for any thoughts you might have on this.

...First: for what it's worth (and it isn't worth much <g>), I agree with what other responders have written.

...Having said that, I'll add a point about which you didn't ask: "my server, my rules." In other words, even if they are wrong about being confident in their spam scoring software, it is the school's / mail provider's right to refuse to deliver / handle any type of e-mail they wish. After all, they paid for the servers and e-mail service and it is their money and resources consumed by the spam.

[oldjack]

...First: for what it's worth (and it isn't worth much <g>), I agree with what other responders have written.

...Having said that, I'll add a point about which you didn't ask: "my server, my rules."

I don't think I implied in my original post that I had any right to tell anyone what to do. If I don't like it, my option is to stop using it (which will be a PITA since I've had the e-mail acct since 1995). But I think I have a right (and maybe even a duty) to suggest that false positives may not be noticed until graduate school acceptances or contractors' bids disappear, and that they can keep the filters but also take steps to help out people who are concerned about false postives.

And it does seem to be a hyperactive spam filter. Only seven spams have made it through to SpamCop in the last three days (vs. 100 spams per day before).

And, although the spam filter is no doubt an excellent product, it's sad that a prestigious university would buy into that "WITHOUT EXCEPTION" stuff.

Whatever, it appears that you can whitelist your regular correspondents and, failing that being a solution, there are "Administrator Customization" options you could negotiate with the school.

Unfortunately, the school won't offer these options to end users.

Thanks to all of you for your thoughts. I really appreciate it!

[turetzsr]

...First: for what it's worth (and it isn't worth much <g>), I agree with what other responders have written.

...Having said that, I'll add a point about which you didn't ask: "my server, my rules."

<snip>

I don't think I implied in my original post that I had any right to tell anyone what to do.

...Quite correct -- you didn't; I didn't mean to imply that I thought that was what you were saying. Sorry! <g> My intended point was only that your school / mail provider have the right to refuse to receive e-mail based on whatever criteria they wish to use.

<snip>

But I think I have a right (and maybe even a duty) to suggest that false positives may not be noticed until graduate school acceptances or contractors' bids disappear, and that they can keep the filters but also take steps to help out people who are concerned about false postives.

<snip>

...Absolutely!

<snip>

it's sad that a prestigious university would buy into that "WITHOUT EXCEPTION" stuff.

<snip>

...Again, absolutely! Of course, they might be just using that as an excuse to keep their servers clear of trash. It's possible that avoiding the spew they would otherwise receive, store and process is worth more than the possible drawbacks. But I hope you can make them see that the drawbacks could be quite serious.