Can you add more information to the Filter Activity Message? I have a bunch of filters that I've added to webmail to move messages to the appropriate folder, but also one filter, specifically for moving known spam to the held mail folder, but this rule sometimes catches legitimate emails...
Right now, the filter message simply says: Filter activity: The message "Message Subject" from "Email Address" has been moved to the folder "Folder Name".
I would like to request that this message be changed to: Filter activity: The message "Message Subject" from "Email Address" has been moved to the folder "Folder Name" because of "Filter Word" in "Name of Rule".
That will make debugging filter rules much easier...
Full Version: Filter Activity Message to include more info
??? perhaps you'd really want to focus on just what and how you are trying to 'filter' something that is causing the problem. Or the usual request for a Tracking URL on the specific item that's causing you grief, in association with your attempted 'filter rule' .... just noting that this isn't an everyday type request. Your description of "known spam" riles that catch 'legitimate e-mail' kind of suggestes that the wrong 'rules' have been written.
QUOTE(Wazoo @ Dec 29 2007, 12:51 AM) 
??? perhaps you'd really want to focus on just what and how you are trying to 'filter' something that is causing the problem. Or the usual request for a Tracking URL on the specific item that's causing you grief, in association with your attempted 'filter rule' .... just noting that this isn't an everyday type request. Your description of "known spam" riles that catch 'legitimate e-mail' kind of suggestes that the wrong 'rules' have been written.
That's possible, but since the Filter message doesn't give much info, it just makes it that much more tedious to debug the rules.. I'm simply asking for the filter message to give more info as to which word and which rule has caused it to filter the message... I'm sure I'm not the only one who would appreciate this, and the requested change should be relatively easy and have no impact on anything else at all.
QUOTE(gregw @ Dec 29 2007, 07:35 PM)
and the requested change should be relatively easy and have no impact on anything else at all.
Sure, OK. I swear I've seen that statement mae elsewhere, typically just prior to everything going bad <g>
I'm not up for yet another Horde/IMP/etc. code search, but just on the face of it, I can see a lot of 'possible impacts' .. in general, most based on that your e-mail isn't the only thing being handled. It sure seems like 'working on your filter' would be better 'for you' ...
QUOTE(Wazoo @ Dec 30 2007, 12:04 AM)
I'm not up for yet another Horde/IMP/etc. code search, but just on the face of it, I can see a lot of 'possible impacts' .. in general, most based on that your e-mail isn't the only thing being handled. It sure seems like 'working on your filter' would be better 'for you' ...
I may be wrong, but I believe what he wants is an improvement to the system to help him do just that.
My understanding of the request is he wants more details in the "yellow bar" which shows what is happening during a filter action. I agree it should have little impact (the exception may be sceen space which is an issue for some in certain configurations, wrapping being one), and I would assume (and all that goes with that word) the values being requested are available as variables in the system.
I'm just having a bit of an issue with trying to guess at just how massive the problem actually is. The list of things that can be filtered against isn't that big, though I'm stuck trying to calculate just how many rules must be in place to lose track of what parameters have been entered ... compounded by that 'catches good e-mail' issue. Trying to contrast this 'filter rule issue looking for bad e-mail and having issues' as compared to re-thinking the whole process and working from a whitelist perspective instead.
I'm still specifically interested in what 'filter rules' have been put into place that causes good e-mail to be mis-handled. In my mind, this is where the actual problem exists .... trying to sort out why a listed 'filtered because it matched' would actually identify a good versus bad e-mail, it seems to me that if there's that circumstance available, then one is still stuck with going through the Held folder looking for good e-mail .. trying to visualize any other way than one e-mail at a time, which to me defeats the whole purpose.
I'm still specifically interested in what 'filter rules' have been put into place that causes good e-mail to be mis-handled. In my mind, this is where the actual problem exists .... trying to sort out why a listed 'filtered because it matched' would actually identify a good versus bad e-mail, it seems to me that if there's that circumstance available, then one is still stuck with going through the Held folder looking for good e-mail .. trying to visualize any other way than one e-mail at a time, which to me defeats the whole purpose.
The problem definitely isn't massive. I just thought that having more detailed info in the "Filter Activity" message would be great for debugging filter rules, and would allow me to know straight away which rule has caused the email to be filtered instead of having to go through each filter rule to know which one actually did the job.
For Example, I have a filter that automatically moves all mails with a link to geocities or googlepages to held mail... I don't know anyone who has a geocities or googlepages web page and 99.99% of emails with such links ARE spam, and are usually not caught by spamcop the first time that they are sent, nor are their spam assassin scores high enough to mark them out as spam. See below for one email that I just received..
For Example, I have a filter that automatically moves all mails with a link to geocities or googlepages to held mail... I don't know anyone who has a geocities or googlepages web page and 99.99% of emails with such links ARE spam, and are usually not caught by spamcop the first time that they are sent, nor are their spam assassin scores high enough to mark them out as spam. See below for one email that I just received..
CODE
Return-Path: <n.quintanakj[at]mail.wplus.net>
Delivered-To: cqmail-net-delete[at]cqmail.net
Received: (qmail 32395 invoked from network); 31 Dec 2007 01:25:06 -0000
X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on filter8
X-Spam-Level:
X-Spam-Status: hits=0.2 tests=URIBL_GREY version=3.2.3
Received: from unknown (192.168.1.108)
by filter8.cesmail.net with QMQP; 31 Dec 2007 01:25:06 -0000
Received: from quan.pair.com (209.68.1.249)
by mx71.cesmail.net with SMTP; 31 Dec 2007 01:25:06 -0000
Received: (qmail 47905 invoked by uid 3112); 31 Dec 2007 00:25:06 -0000
Delivered-To: delete
Received: (qmail 47902 invoked from network); 31 Dec 2007 00:25:06 -0000
Received: from mailwash12.pair.com (66.39.2.12)
by quan.pair.com with SMTP; 31 Dec 2007 00:25:06 -0000
Received: from localhost (localhost [127.0.0.1])
by mailwash12.pair.com (Postfix) with SMTP id 646D267595
for <delete>; Sun, 30 Dec 2007 19:25:06 -0500 (EST)
Received: from cnh.com (201.139.130.84.cable.dyn.cableonline.com.mx [201.139.130.84])
by mailwash12.pair.com (Postfix) with ESMTP id 36DB86758D
for <delete>; Sun, 30 Dec 2007 19:25:03 -0500 (EST)
Reply-To: "Natasha Quintana" <n.quintanakj[at]mail.wplus.net>
Subject: GucciRolex..CheapPriceWatches..Omega, Breitling, Bvlgari for you 7bwc4u2ogk74o
Sender: <n.quintanakj[at]mail.wplus.net>
X-Sender: <n.quintanakj[at]mail.wplus.net>
To: delete
From: "Natasha Quintana" <n.quintanakj[at]mail.wplus.net>
Message-ID: <1199066725.1872[at]mail.wplus.net>
In-Reply-To: <7eb401c84aee$65a5e7e2$c6458296[at]pw73e44>
Date: Sun, 30 Dec 2007 19:05:25 -0700
Content-Type: text/plain;
charset="iso-8859-2"
Content-Transfer-Encoding: 8bit
X-SpamCop-Checked: 209.68.1.249 66.39.2.12 201.139.130.84
:: New Arrival 2007 models ::
RolexMens
RolexLadies
RolexSports
RolexDateJusts
A.Lange & Sohne
Alain Silberstein
Audemars Piguet
Bell & Ross
Breguet
Breitling
Bvlgari
Cartier
Chanel
Chopard
Chronoswiss
Corum
Franck Muller
Glashutte
Gucci
Hermes
Hublot
IWC
Jacob & Co
Jaeger-Lecoultre
Longines
Louis Vuitton
Mont Blanc
Movado
Omega
Oris
Panerai
Patek Philippe
Porsche Design
Rado
Roger Dubuis
SWISS Rolex
Tag Heuer
Technomarine
Vacheron Constantin
Zenith
Order Your Brand New Watches Now!
http://zokysiny95408.googlepages.com/index.html
Delivered-To: cqmail-net-delete[at]cqmail.net
Received: (qmail 32395 invoked from network); 31 Dec 2007 01:25:06 -0000
X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on filter8
X-Spam-Level:
X-Spam-Status: hits=0.2 tests=URIBL_GREY version=3.2.3
Received: from unknown (192.168.1.108)
by filter8.cesmail.net with QMQP; 31 Dec 2007 01:25:06 -0000
Received: from quan.pair.com (209.68.1.249)
by mx71.cesmail.net with SMTP; 31 Dec 2007 01:25:06 -0000
Received: (qmail 47905 invoked by uid 3112); 31 Dec 2007 00:25:06 -0000
Delivered-To: delete
Received: (qmail 47902 invoked from network); 31 Dec 2007 00:25:06 -0000
Received: from mailwash12.pair.com (66.39.2.12)
by quan.pair.com with SMTP; 31 Dec 2007 00:25:06 -0000
Received: from localhost (localhost [127.0.0.1])
by mailwash12.pair.com (Postfix) with SMTP id 646D267595
for <delete>; Sun, 30 Dec 2007 19:25:06 -0500 (EST)
Received: from cnh.com (201.139.130.84.cable.dyn.cableonline.com.mx [201.139.130.84])
by mailwash12.pair.com (Postfix) with ESMTP id 36DB86758D
for <delete>; Sun, 30 Dec 2007 19:25:03 -0500 (EST)
Reply-To: "Natasha Quintana" <n.quintanakj[at]mail.wplus.net>
Subject: GucciRolex..CheapPriceWatches..Omega, Breitling, Bvlgari for you 7bwc4u2ogk74o
Sender: <n.quintanakj[at]mail.wplus.net>
X-Sender: <n.quintanakj[at]mail.wplus.net>
To: delete
From: "Natasha Quintana" <n.quintanakj[at]mail.wplus.net>
Message-ID: <1199066725.1872[at]mail.wplus.net>
In-Reply-To: <7eb401c84aee$65a5e7e2$c6458296[at]pw73e44>
Date: Sun, 30 Dec 2007 19:05:25 -0700
Content-Type: text/plain;
charset="iso-8859-2"
Content-Transfer-Encoding: 8bit
X-SpamCop-Checked: 209.68.1.249 66.39.2.12 201.139.130.84
:: New Arrival 2007 models ::
RolexMens
RolexLadies
RolexSports
RolexDateJusts
A.Lange & Sohne
Alain Silberstein
Audemars Piguet
Bell & Ross
Breguet
Breitling
Bvlgari
Cartier
Chanel
Chopard
Chronoswiss
Corum
Franck Muller
Glashutte
Gucci
Hermes
Hublot
IWC
Jacob & Co
Jaeger-Lecoultre
Longines
Louis Vuitton
Mont Blanc
Movado
Omega
Oris
Panerai
Patek Philippe
Porsche Design
Rado
Roger Dubuis
SWISS Rolex
Tag Heuer
Technomarine
Vacheron Constantin
Zenith
Order Your Brand New Watches Now!
http://zokysiny95408.googlepages.com/index.html
QUOTE(gregw @ Dec 30 2007, 07:43 PM)
For Example, I have a filter that automatically moves all mails with a link to geocities or googlepages to held mail... I don't know anyone who has a geocities or googlepages web page and 99.99% of emails with such links ARE spam, and are usually not caught by spamcop the first time that they are sent, nor are their spam assassin scores high enough to mark them out as spam. See below for one email that I just received..
Heres the catch .. I asked for an example of a 'rule' you are using that tripped up on a 'good' e-mail. All I can make out of your last is that you 'suggest' that you have made up some kind of rule (not defined here) that looks for some body content ... then provide an e-mail "you just received" ....???? It sure looks like spam to me, but you don't say whether it was "caught by your filter and moved to the Held folder" ... remained in your InBox ... or if anything at all happened. ?????
From the Original/Official FAQ ..... How can this kind of message get thru your filter? ..... Here's how the system works. We don't look at the content of the message. It takes too long and we can't scale that to 10s of thousands of users.....
(Gosh, please don't suggest that the Original/Official FAQ might be wrong/be out-of-date/contains bad data!)
There's an ancient bit of an attempted FAQ entry 'here' that left me confused (not having a spamcop.net e-mail account) .... FAQ about the personal webmail filters, Client filters within webmail
Another attempted FAQ 'here' Messages not Filtered - Why? doesn't actually 'directly' address the 'same' filtering as involved here.
Note StevenUnderwood's Reply in Messages not Filtered - Why? going along with my suggested alternative of filtering via the whitelist approach.
Hmmmm, words about modification of code, suggesting (yet again) that even looking at the Horde/IMP codebase repository might not 'explain' how things work on JT's servers, and of course noting that this was from a few years back ... Applying filters, having problems with filters not working
At this point, I've blown the better part of the last 1.5 hours going through various Google search results trying to find something 'like' this in previous Topics/Discussions ... read a ton-load of stuff, but .. no score. Almost everything I looked at really dealt with spam not being filtered, nothing with 'good' e-mail being caught because of a filter setting.
How much clearer do I have to be... 
Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities, after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...
I don't know how you got the impression that I was complaining about such an email getting through the spam filters...
Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities, after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...
I don't know how you got the impression that I was complaining about such an email getting through the spam filters...
QUOTE(gregw @ Dec 31 2007, 08:05 AM)
Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities, after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...
You have not explained exactly what you are doing to the filter once it catches a false positive.
If you are using "keywords" to filter spam, there will ALWAYS be false positives unless the only non-spam messages are canned messages where there are no unexpected words.
Modifying a filter each time a message is accidentally caught is likely to be a waste of time, unless you expect to receive more of the same, in which case, deleting that filter is probably the best solution, because it is no longer effective.
Now, I could see where having the email address of the sender posted in the filter message may help to determine if there are false positives in the Held Mail folder, helping you not to report them, but that is not your stated reason for this change.
QUOTE(gregw @ Dec 31 2007, 07:05 AM)
How much clearer do I have to be...
I don't know. I'm getting the feeling of beating my head against the wall in my attempt to try to help you sort out this filtering 'problem' ....How much clearer do I need to be?
QUOTE
Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities,
OK, again, there are words offered to 'describe' your rules, but the rule itself (and action) are still actually undefined. If one was to take what you just said literally, the 'filter effect' would probably be next to nil, as the amount of e-mail with "Body Contains geocities AND googlepages" would be pretty danged small.
Note: I have asked for both "the rule" and "the caught good e-mail" for work examples. Provided was an example of a spam caught by an undefined rule. Then words used to describe the 'good e-mail caught' scenario.
Your friend's e-mail getting caught by one of your filters leads us/me back to the whitelisting aspect. Have you whitelisted your friend's e-mail address?
QUOTE
after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...
All I'm suggesting that in the case you've laid out, It would seem pretty obvious that your friend's e-mail included keywords that you defined in your filter set. Where I'm coming from, I'm not really graspiong what having having an 'expanded' listing of the filter actions involved at the time of the filter application would really do to 'help' in this case. This coming from trying to imaging one line amongst several hundred that actully signified that a good e-mail had been 'filtered' amongst all the spam.
As noted by Steven and expanded to the real world, filtering on 'words' has been a problem since the beginning. Your offered example isn't much different than those web-filtering kits that wouldn't let anything through with the word "breast" involved, thinking that this would keep kids from stumbling into that nasty sex stuff .... but the result was women couldn't do any research on 'breast cancer' or mascectomy ... recipes involving certain parts of a chicken or turkey couldn't be displayed ... all unintended results of filtering on specific words.
QUOTE
I don't know how you got the impression that I was complaining about such an email getting through the spam filters...
I have no idea where you picked up that thought .. my queries have been about good e-mail caught by your filters which is what you started out with. I then tried to suggest reversing the thought process to think of whitelisting the good suff rather than going nuts trying to find the magic words to block all the bad stuff.
QUOTE(Wazoo @ Dec 31 2007, 12:03 PM)
<snip>
All I'm suggesting that in the case you've laid out, It would seem pretty obvious that your friend's e-mail included keywords that you defined in your filter set. Where I'm coming from, I'm not really graspiong what having having an 'expanded' listing of the filter actions involved at the time of the filter application would really do to 'help' in this case.
<snip>
...IIUC, the OP (gregw) has given an example of a case where (s)he has been able to determine, with some pain, the filter that was causing the problem and is asking for a change that would enable her/him to find the problem more easily. All I'm suggesting that in the case you've laid out, It would seem pretty obvious that your friend's e-mail included keywords that you defined in your filter set. Where I'm coming from, I'm not really graspiong what having having an 'expanded' listing of the filter actions involved at the time of the filter application would really do to 'help' in this case.
<snip>
...gregw, do I have that right? If so, I don't see much utility in Wazoo and StevenUnderwood offering further input or inquiries (although I think it may have been worth the attempt up to about linear post #9
).
QUOTE(turetzsr @ Dec 31 2007, 05:30 PM)
...IIUC, the OP (gregw) has given an example of a case where (s)he has been able to determine, with some pain, the filter that was causing the problem and is asking for a change that would enable her/him to find the problem more easily.
...gregw, do I have that right? If so, I don't see much utility in Wazoo and StevenUnderwood offering further input or inquiries (although I think it may have been worth the attempt up to about linear post #9 ().
...gregw, do I have that right? If so, I don't see much utility in Wazoo and StevenUnderwood offering further input or inquiries (although I think it may have been worth the attempt up to about linear post #9 (
).You got it right!
QUOTE(gregw @ Dec 31 2007, 08:02 PM)
You got it right!
OK, 'scuze the hell out of me for even thinking about trying to work out and resolve the 'real' issue.
OK my suggestion.
Add logic to your filters to also test for X-SpamCop-Whitelisted: and not filter if it is present
Thus if there is a problem you can always get out of it (exactly as with Block lists and SpamAssassin)
by whitelisting your friend even if you can't figure out what is happening.
BTW, my major problem with webmail filters was when they didn't do anything and I couldn't figure out why.
I normally test them by copying just one or two emails to a folder by themselves and disabling and enabling filters until I found the problem. If you have 32 filters this should take no more than 5 trials.
Best of luck.
Add logic to your filters to also test for X-SpamCop-Whitelisted: and not filter if it is present
Thus if there is a problem you can always get out of it (exactly as with Block lists and SpamAssassin)
by whitelisting your friend even if you can't figure out what is happening.
BTW, my major problem with webmail filters was when they didn't do anything and I couldn't figure out why.
I normally test them by copying just one or two emails to a folder by themselves and disabling and enabling filters until I found the problem. If you have 32 filters this should take no more than 5 trials.
Best of luck.
QUOTE(Wazoo @ Jan 1 2008, 01:49 AM)
OK, 'scuze the hell out of me for even thinking about trying to work out and resolve the 'real' issue.
...No biggie, at least as far as I'm concerned. Note, however, that this is the "New Feature Request" forum and, therefore, it may be that a discussion of the 'real' issue might have been more appropriate (as well as less controversial and also less confusing to the OP and other readers) in another SpamCop forum. <g>
QUOTE(turetzsr @ Jan 2 2008, 07:07 PM)
...No biggie, at least as far as I'm concerned.
That reply was not made in reference to your comment.
QUOTE
Note, however, that this is the "New Feature Request" forum and, therefore, it may be that a discussion of the 'real' issue might have been more appropriate (as well as less controversial and also less confusing to the OP and other readers) in another SpamCop forum. <g>
<g>I'm still struggling with the utility of the request, based on the data provided thus far, complicated by the lack of answers to questions asked.
As 'defined' thus far, the request would have one line 'of interest' embedded within several hundred others ..
e-mail from disneydumper was moved to the Held folder because it included the word Geocities
As asked in a couple of different ways by different people, exactly what "troubleshooting" steps would then be involved .. and to what end? The use of "magic words" as filters has been addressed within this Topic as well as all over the Internet over the years. That one cannot control what a friend will include in a future e-mail only compounds the downside to this type of configuration. The implication seems to be that a single e-mail from a friend would be reason enough to remove one of those 'magic words' from the list, yet per the same description, the door is now opened to receiving those other hundreds of spams-a-day that were blocked 'yesterday' ....
Yes, I thought about moving it several times, but .. kept thinking that the real descision point was 'just around the corner' .... for example, once we got the "whitelisting of my friend's address solved the problem" scenario resolved or broken ....
Thanks, folks. This thread gave me some ideas to "tune" my Spamcop filters, and I think it's solved several of my "spam not caught" problems.
I'm fortunate that I have no desire to receive mail about or from (for example) Viagra, Pfizer, or casinos, even if it is from people I know.
Next step is filters that look for \ / instead of V, and other such deliberate misspellings.
I'm fortunate that I have no desire to receive mail about or from (for example) Viagra, Pfizer, or casinos, even if it is from people I know.
Next step is filters that look for \ / instead of V, and other such deliberate misspellings.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.