Jump to content
Sign in to follow this  
gregw

Filter Activity Message to include more info

Recommended Posts

Can you add more information to the Filter Activity Message? I have a bunch of filters that I've added to webmail to move messages to the appropriate folder, but also one filter, specifically for moving known spam to the held mail folder, but this rule sometimes catches legitimate emails...

Right now, the filter message simply says: Filter activity: The message "Message Subject" from "Email Address" has been moved to the folder "Folder Name".

I would like to request that this message be changed to: Filter activity: The message "Message Subject" from "Email Address" has been moved to the folder "Folder Name" because of "Filter Word" in "Name of Rule".

That will make debugging filter rules much easier...

Share this post


Link to post
Share on other sites

??? perhaps you'd really want to focus on just what and how you are trying to 'filter' something that is causing the problem. Or the usual request for a Tracking URL on the specific item that's causing you grief, in association with your attempted 'filter rule' .... just noting that this isn't an everyday type request. Your description of "known spam" riles that catch 'legitimate e-mail' kind of suggestes that the wrong 'rules' have been written.

Share this post


Link to post
Share on other sites
??? perhaps you'd really want to focus on just what and how you are trying to 'filter' something that is causing the problem. Or the usual request for a Tracking URL on the specific item that's causing you grief, in association with your attempted 'filter rule' .... just noting that this isn't an everyday type request. Your description of "known spam" riles that catch 'legitimate e-mail' kind of suggestes that the wrong 'rules' have been written.

That's possible, but since the Filter message doesn't give much info, it just makes it that much more tedious to debug the rules.. I'm simply asking for the filter message to give more info as to which word and which rule has caused it to filter the message... I'm sure I'm not the only one who would appreciate this, and the requested change should be relatively easy and have no impact on anything else at all.

Share this post


Link to post
Share on other sites
and the requested change should be relatively easy and have no impact on anything else at all.

Sure, OK. I swear I've seen that statement mae elsewhere, typically just prior to everything going bad <g>

I'm not up for yet another Horde/IMP/etc. code search, but just on the face of it, I can see a lot of 'possible impacts' .. in general, most based on that your e-mail isn't the only thing being handled. It sure seems like 'working on your filter' would be better 'for you' ...

Share this post


Link to post
Share on other sites

I'm not up for yet another Horde/IMP/etc. code search, but just on the face of it, I can see a lot of 'possible impacts' .. in general, most based on that your e-mail isn't the only thing being handled. It sure seems like 'working on your filter' would be better 'for you' ...

I may be wrong, but I believe what he wants is an improvement to the system to help him do just that.

My understanding of the request is he wants more details in the "yellow bar" which shows what is happening during a filter action. I agree it should have little impact (the exception may be sceen space which is an issue for some in certain configurations, wrapping being one), and I would assume (and all that goes with that word) the values being requested are available as variables in the system.

Share this post


Link to post
Share on other sites

I'm just having a bit of an issue with trying to guess at just how massive the problem actually is. The list of things that can be filtered against isn't that big, though I'm stuck trying to calculate just how many rules must be in place to lose track of what parameters have been entered ... compounded by that 'catches good e-mail' issue. Trying to contrast this 'filter rule issue looking for bad e-mail and having issues' as compared to re-thinking the whole process and working from a whitelist perspective instead.

I'm still specifically interested in what 'filter rules' have been put into place that causes good e-mail to be mis-handled. In my mind, this is where the actual problem exists .... trying to sort out why a listed 'filtered because it matched' would actually identify a good versus bad e-mail, it seems to me that if there's that circumstance available, then one is still stuck with going through the Held folder looking for good e-mail .. trying to visualize any other way than one e-mail at a time, which to me defeats the whole purpose.

Share this post


Link to post
Share on other sites

The problem definitely isn't massive. I just thought that having more detailed info in the "Filter Activity" message would be great for debugging filter rules, and would allow me to know straight away which rule has caused the email to be filtered instead of having to go through each filter rule to know which one actually did the job.

For Example, I have a filter that automatically moves all mails with a link to geocities or googlepages to held mail... I don't know anyone who has a geocities or googlepages web page and 99.99% of emails with such links ARE spam, and are usually not caught by spamcop the first time that they are sent, nor are their spam assassin scores high enough to mark them out as spam. See below for one email that I just received..

Return-Path: &lt;n.quintanakj[at]mail.wplus.net&gt;
Delivered-To: cqmail-net-delete[at]cqmail.net
Received: (qmail 32395 invoked from network); 31 Dec 2007 01:25:06 -0000
X-spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on filter8
X-spam-Level: 
X-spam-Status: hits=0.2 tests=URIBL_GREY version=3.2.3
Received: from unknown (192.168.1.108)
  by filter8.cesmail.net with QMQP; 31 Dec 2007 01:25:06 -0000
Received: from quan.pair.com (209.68.1.249)
  by mx71.cesmail.net with SMTP; 31 Dec 2007 01:25:06 -0000
Received: (qmail 47905 invoked by uid 3112); 31 Dec 2007 00:25:06 -0000
Delivered-To: delete
Received: (qmail 47902 invoked from network); 31 Dec 2007 00:25:06 -0000
Received: from mailwash12.pair.com (66.39.2.12)
  by quan.pair.com with SMTP; 31 Dec 2007 00:25:06 -0000
Received: from localhost (localhost [127.0.0.1])
	by mailwash12.pair.com (Postfix) with SMTP id 646D267595
	for &lt;delete&gt;; Sun, 30 Dec 2007 19:25:06 -0500 (EST)
Received: from cnh.com (201.139.130.84.cable.dyn.cableonline.com.mx [201.139.130.84])
	by mailwash12.pair.com (Postfix) with ESMTP id 36DB86758D
	for &lt;delete&gt;; Sun, 30 Dec 2007 19:25:03 -0500 (EST)
Reply-To: "Natasha Quintana" &lt;n.quintanakj[at]mail.wplus.net&gt;
Subject: GucciRolex..CheapPriceWatches..Omega, Breitling, Bvlgari for you 7bwc4u2ogk74o
Sender: &lt;n.quintanakj[at]mail.wplus.net&gt;
X-Sender: &lt;n.quintanakj[at]mail.wplus.net&gt;
To: delete
From: "Natasha Quintana" &lt;n.quintanakj[at]mail.wplus.net&gt;
Message-ID: &lt;1199066725.1872[at]mail.wplus.net&gt;
In-Reply-To: &lt;7eb401c84aee$65a5e7e2$c6458296[at]pw73e44&gt;
Date: Sun, 30 Dec 2007 19:05:25 -0700
Content-Type: text/plain;
	charset="iso-8859-2"
Content-Transfer-Encoding: 8bit
X-SpamCop-Checked: 209.68.1.249 66.39.2.12 201.139.130.84 

:: New Arrival 2007 models ::		   

RolexMens	
RolexLadies	   
RolexSports	  
RolexDateJusts		  
A.Lange &amp; Sohne		  
Alain Silberstein  
Audemars Piguet		 
Bell &amp; Ross	   
Breguet		 
Breitling  
Bvlgari	  
Cartier		 
Chanel 
Chopard   
Chronoswiss   
Corum  
Franck Muller  
Glashutte   
Gucci   
Hermes 
Hublot	 
IWC
Jacob &amp; Co	
Jaeger-Lecoultre	   
Longines	  
Louis Vuitton	  
Mont Blanc	
Movado  
Omega   
Oris  
Panerai   
Patek Philippe  
Porsche Design
Rado	  
Roger Dubuis
SWISS Rolex  
Tag Heuer	   
Technomarine  
Vacheron Constantin	   
Zenith	   

Order Your Brand New Watches Now!	  

http://zokysiny95408.googlepages.com/index.html

Share this post


Link to post
Share on other sites
For Example, I have a filter that automatically moves all mails with a link to geocities or googlepages to held mail... I don't know anyone who has a geocities or googlepages web page and 99.99% of emails with such links ARE spam, and are usually not caught by spamcop the first time that they are sent, nor are their spam assassin scores high enough to mark them out as spam. See below for one email that I just received..

Heres the catch .. I asked for an example of a 'rule' you are using that tripped up on a 'good' e-mail. All I can make out of your last is that you 'suggest' that you have made up some kind of rule (not defined here) that looks for some body content ... then provide an e-mail "you just received" ....???? It sure looks like spam to me, but you don't say whether it was "caught by your filter and moved to the Held folder" ... remained in your InBox ... or if anything at all happened. ?????

From the Original/Official FAQ ..... How can this kind of message get thru your filter? ..... Here's how the system works. We don't look at the content of the message. It takes too long and we can't scale that to 10s of thousands of users.....

(Gosh, please don't suggest that the Original/Official FAQ might be wrong/be out-of-date/contains bad data!)

There's an ancient bit of an attempted FAQ entry 'here' that left me confused (not having a spamcop.net e-mail account) .... FAQ about the personal webmail filters, Client filters within webmail

Another attempted FAQ 'here' Messages not Filtered - Why? doesn't actually 'directly' address the 'same' filtering as involved here.

Note StevenUnderwood's Reply in Messages not Filtered - Why? going along with my suggested alternative of filtering via the whitelist approach.

Hmmmm, words about modification of code, suggesting (yet again) that even looking at the Horde/IMP codebase repository might not 'explain' how things work on JT's servers, and of course noting that this was from a few years back ... Applying filters, having problems with filters not working

At this point, I've blown the better part of the last 1.5 hours going through various Google search results trying to find something 'like' this in previous Topics/Discussions ... read a ton-load of stuff, but .. no score. Almost everything I looked at really dealt with spam not being filtered, nothing with 'good' e-mail being caught because of a filter setting.

Share this post


Link to post
Share on other sites

How much clearer do I have to be... :rolleyes:

Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities, after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...

I don't know how you got the impression that I was complaining about such an email getting through the spam filters... <_<

Share this post


Link to post
Share on other sites

Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities, after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...

You have not explained exactly what you are doing to the filter once it catches a false positive.

If you are using "keywords" to filter spam, there will ALWAYS be false positives unless the only non-spam messages are canned messages where there are no unexpected words.

Modifying a filter each time a message is accidentally caught is likely to be a waste of time, unless you expect to receive more of the same, in which case, deleting that filter is probably the best solution, because it is no longer effective.

Now, I could see where having the email address of the sender posted in the filter message may help to determine if there are false positives in the Held Mail folder, helping you not to report them, but that is not your stated reason for this change.

Share this post


Link to post
Share on other sites
How much clearer do I have to be... :rolleyes:

I don't know. I'm getting the feeling of beating my head against the wall in my attempt to try to help you sort out this filtering 'problem' ....How much clearer do I need to be?

Yes, the above example was caught by my filter rule and filtered to the "Held Mail" folder. This rule, using the keywords, "geocities" and "googlepages", also caught a legitimate email once, because a friend actually wanted me to see a link from geocities,

OK, again, there are words offered to 'describe' your rules, but the rule itself (and action) are still actually undefined. If one was to take what you just said literally, the 'filter effect' would probably be next to nil, as the amount of e-mail with "Body Contains geocities AND googlepages" would be pretty danged small.

Note: I have asked for both "the rule" and "the caught good e-mail" for work examples. Provided was an example of a spam caught by an undefined rule. Then words used to describe the 'good e-mail caught' scenario.

Your friend's e-mail getting caught by one of your filters leads us/me back to the whitelisting aspect. Have you whitelisted your friend's e-mail address?

after filtering many hundreds of spam directly to the held mail box, and a more detailed message when filtering might have made it easier for me to find which rule actually did the filtering...

All I'm suggesting that in the case you've laid out, It would seem pretty obvious that your friend's e-mail included keywords that you defined in your filter set. Where I'm coming from, I'm not really graspiong what having having an 'expanded' listing of the filter actions involved at the time of the filter application would really do to 'help' in this case. This coming from trying to imaging one line amongst several hundred that actully signified that a good e-mail had been 'filtered' amongst all the spam.

As noted by Steven and expanded to the real world, filtering on 'words' has been a problem since the beginning. Your offered example isn't much different than those web-filtering kits that wouldn't let anything through with the word "breast" involved, thinking that this would keep kids from stumbling into that nasty sex stuff .... but the result was women couldn't do any research on 'breast cancer' or mascectomy ... recipes involving certain parts of a chicken or turkey couldn't be displayed ... all unintended results of filtering on specific words.

I don't know how you got the impression that I was complaining about such an email getting through the spam filters... <_<

I have no idea where you picked up that thought .. my queries have been about good e-mail caught by your filters which is what you started out with. I then tried to suggest reversing the thought process to think of whitelisting the good suff rather than going nuts trying to find the magic words to block all the bad stuff.

Share this post


Link to post
Share on other sites
<snip>

All I'm suggesting that in the case you've laid out, It would seem pretty obvious that your friend's e-mail included keywords that you defined in your filter set. Where I'm coming from, I'm not really graspiong what having having an 'expanded' listing of the filter actions involved at the time of the filter application would really do to 'help' in this case.

<snip>

...IIUC, the OP (gregw) has given an example of a case where (s)he has been able to determine, with some pain, the filter that was causing the problem and is asking for a change that would enable her/him to find the problem more easily.

...gregw, do I have that right? If so, I don't see much utility in Wazoo and StevenUnderwood offering further input or inquiries (although I think it may have been worth the attempt up to about linear post #9 61844[/snapback]).

Share this post


Link to post
Share on other sites
...IIUC, the OP (gregw) has given an example of a case where (s)he has been able to determine, with some pain, the filter that was causing the problem and is asking for a change that would enable her/him to find the problem more easily.

...gregw, do I have that right? If so, I don't see much utility in Wazoo and StevenUnderwood offering further input or inquiries (although I think it may have been worth the attempt up to about linear post #9 (61844[/snapback]).

You got it right! :)

Share this post


Link to post
Share on other sites
You got it right! :)

OK, 'scuze the hell out of me for even thinking about trying to work out and resolve the 'real' issue.

Share this post


Link to post
Share on other sites

OK my suggestion.

Add logic to your filters to also test for X-SpamCop-Whitelisted: and not filter if it is present

Thus if there is a problem you can always get out of it (exactly as with Block lists and SpamAssassin)

by whitelisting your friend even if you can't figure out what is happening.

BTW, my major problem with webmail filters was when they didn't do anything and I couldn't figure out why.

I normally test them by copying just one or two emails to a folder by themselves and disabling and enabling filters until I found the problem. If you have 32 filters this should take no more than 5 trials.

Best of luck.

Share this post


Link to post
Share on other sites
OK, 'scuze the hell out of me for even thinking about trying to work out and resolve the 'real' issue.
...No biggie, at least as far as I'm concerned. Note, however, that this is the "New Feature Request" forum and, therefore, it may be that a discussion of the 'real' issue might have been more appropriate (as well as less controversial and also less confusing to the OP and other readers) in another SpamCop forum. :) <g>

Share this post


Link to post
Share on other sites
...No biggie, at least as far as I'm concerned.

That reply was not made in reference to your comment.

Note, however, that this is the "New Feature Request" forum and, therefore, it may be that a discussion of the 'real' issue might have been more appropriate (as well as less controversial and also less confusing to the OP and other readers) in another SpamCop forum. :) <g>

I'm still struggling with the utility of the request, based on the data provided thus far, complicated by the lack of answers to questions asked.

As 'defined' thus far, the request would have one line 'of interest' embedded within several hundred others ..

e-mail from disneydumper was moved to the Held folder because it included the word Geocities

As asked in a couple of different ways by different people, exactly what "troubleshooting" steps would then be involved .. and to what end? The use of "magic words" as filters has been addressed within this Topic as well as all over the Internet over the years. That one cannot control what a friend will include in a future e-mail only compounds the downside to this type of configuration. The implication seems to be that a single e-mail from a friend would be reason enough to remove one of those 'magic words' from the list, yet per the same description, the door is now opened to receiving those other hundreds of spams-a-day that were blocked 'yesterday' ....

Yes, I thought about moving it several times, but .. kept thinking that the real descision point was 'just around the corner' .... for example, once we got the "whitelisting of my friend's address solved the problem" scenario resolved or broken ....

Share this post


Link to post
Share on other sites

Thanks, folks. This thread gave me some ideas to "tune" my Spamcop filters, and I think it's solved several of my "spam not caught" problems.

I'm fortunate that I have no desire to receive mail about or from (for example) Viagra, Pfizer, or casinos, even if it is from people I know.

Next step is filters that look for \ / instead of V, and other such deliberate misspellings.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×