Problem: hosts with multiple servers

[DavidT 0]

Mail for one of my domains is filtered by an "Email Defense" service, so I carefully ran a SpamCop mailhosts probe through that system and got it added to my mailhosts config. When looking at the entry in my mailhosts, I see this:

emaildefenseservice.com

with a drop-down containing these hosts:

c1m5.emaildefenseservice.com

emaildefenseservice.com

c1m1.emaildefenseservice.com

fr3.webmaillogin.com

fr4.webmaillogin.com

webmaillogin.com

fr1.webmaillogin.com

fr2.webmaillogin.com

fr5.webmaillogin.com

c1m4.emaildefenseservice.com

and a list of IP addresses. However, looking at that list, you can probably guess that these two hosts probably also exist:

c1m2.emaildefenseservice.com

c1m3.emaildefenseservice.com

Well...they do, and in a batch of Held Mail that I Quick Reported were two spams sent to a webmaster address that both passed through "c1m2.emaildefenseservice.com" which the parser didn't associate with the Email Defense hosts, so the reports wound up getting sent to my own hosting provider!

Here are the problematic lines from the parsing:

Hostname verified: c1m2.emaildefenseservice.com

Possible forgery. Supposed receiving system not associated with any of your mailhosts

Will not trust anything beyond this header

Where does SpamCop get the list of hosts when there are multiple hosts like this? If they took a look at the Senderbase data on "emaildefenseservice.com," they would have identified all of them:

http://www.senderbase.org/search?searchStr...enseservice.com

Senderbase = Ironport = Spamcop!

I've sent a report to my host's Abuse dept and to the SC deputies address. I'm posting here to show that the system is NOT quite ready for "prime time." If we were allowed to directly edit our mailhosts, I could have added those missing hosts manually, and this wouldn't have happened.

DT

Edited September 18, 2004 by DavidT

[Ellen 0]

Mail for one of my domains is filtered by an "Email Defense" service, so I carefully ran a SpamCop mailhosts probe through that system and got it added to my mailhosts config. When looking at the entry in my mailhosts, I see this:

emaildefenseservice.com

with a drop-down containing these hosts:

c1m5.emaildefenseservice.com

emaildefenseservice.com

c1m1.emaildefenseservice.com

fr3.webmaillogin.com

fr4.webmaillogin.com

webmaillogin.com

fr1.webmaillogin.com

fr2.webmaillogin.com

fr5.webmaillogin.com

c1m4.emaildefenseservice.com

and a list of IP addresses. However, looking at that list, you can probably guess that these two hosts probably also exist:

c1m2.emaildefenseservice.com

c1m3.emaildefenseservice.com

Well...they do, and in a batch of Held Mail that I Quick Reported were two spams sent to a webmaster address that both passed through "c1m2.emaildefenseservice.com" which the parser didn't associate with the Email Defense hosts, so the reports wound up getting sent to my own hosting provider!  GRRRRRRR.....I went through all this mailhosts hell to AVOID that!!!!  GRRRRRRR

Here are the problematic lines from the parsing:

Hostname verified: c1m2.emaildefenseservice.com

Possible forgery. Supposed receiving system not associated with any of your mailhosts

Will not trust anything beyond this header

Where does SpamCop get the list of hosts when there are multiple hosts like this? If they took a look at the Senderbase data on "emaildefenseservice.com," they would have identified all of them:

http://www.senderbase.org/search?searchStr...enseservice.com

Senderbase = Ironport = Spamcop!

I've sent a report to my host's Abuse dept and to the SC deputies address. I'm posting here to show that the system is NOT quite ready for "prime time." If we were allowed to directly edit our mailhosts, I could have added those missing hosts manually, and this wouldn't have happened.

DT

17260[/snapback]

Without your registered SC email address I cannot look up the original probe and see what SC saw. And BTW senderbase is not spamcop or vice versa. We are owned by Ironport but are not senderbase.

Ellen

[DavidT 0]

Without your registered SC email address I cannot look up the original probe and see what SC saw.

Good point....I forgot to include that in the apology I sent to my host that I CC'd to the deputies. I'll send it to the deputies address as a followup.

And BTW senderbase is not spamcop or vice versa. We are owned by Ironport but are not senderbase.

My mention of that was too simplistic. I meant to say that it's all owned by Ironport, so all part of the same "family." Both Senderbase and SpamCop belong to Ironport, so my point was that if there's detailed host data in Senderbase, it should be accessible and used during the SpamCop mailhosts definition process....which doesn't seem to be happening.

DT

[DavidT 0]

Further on this topic:

In order to research the depth of this potential problem with the configuration of mailhosts for domains with multiple hosts, I displayed my own "Mailhosts" list, grabbed the HTML code of that page, copied the "drop-down" list for Yahoo, edited out everything but the host names, and proceeded to sort it all into a logical order (partly using MS Word's table sorting capabilities, but mostly by hand).

I came up with a list of 488 different hosts that the SpamCop parser might associate with email coming from the Yahoo system, and I found obvious gaps, where the mailhosts system doesn't "know" about some servers that are valid mail hosts for Yahoo.

For example, here's a small portion of one of the primary categories:

mta100.mail.dcn.yahoo.com

mta101.mail.dcn.yahoo.com

mta102.mail.dcn.yahoo.com

mta104.mail.dcn.yahoo.com

mta106.mail.dcn.yahoo.com

mta108.mail.dcn.yahoo.com

mta109.mail.dcn.yahoo.com

mta110.mail.dcn.yahoo.com

mta111.mail.dcn.yahoo.com

mta114.mail.dcn.yahoo.com

mta115.mail.dcn.yahoo.com

You'll note that the servers numbered 103, 105, 107, and 112 are missing from that sequence, but do a Google search on any of those server names (be sure to enclose them in quotes) and you'll see that they all exist. Furthermore, the first search "hit" in most of the cases is a Senderbase list of Yahoo hosts:

http://www.senderbase.org/?searchBy=organi...=hostname%20asc

So, my point is that the current SC mailhosts registration process should be using the server data from Senderbase when populating the lists of "Hosts/Domains" associated with each mailhost. If that were the case, then the problem I had this morning, where one of the hosts for "emaildefenseservice.com" wasn't registered, wouldn't have happened.

DT

[StevenUnderwood 0]

it should be accessible and used during the SpamCop mailhosts definition process....which doesn't seem to be happening.

The mailhost data is generated by what the MXes are for a certain host and the hosts seen in the travels of the probe message to all of those MXes. Additions are made when another probe goes through a comon host but different servers are seen.

If you want an accurate list, you can NOT simply add a host because it numerically is in line. That host could have a configuration problem and not show the headers properly, for instance.

This is the same reason, in my opinion, you will not see individually editable mailhost configurations. You can always ssend an additional probe fit he configuration has changed.

[DavidT 0]

The mailhost data is generated by what the MXes are for a certain host and the hosts seen in the travels of the probe message to all of those MXes.  Additions are made when another probe goes through a comon host but different servers are seen.

But that's haphazard and random, and not a very thorough way to do things with the bigger email systems.

If you want an accurate list, you can NOT simply add a host because it numerically is in line. That host could have a configuration problem and not show the headers properly, for instance.

I didn't say that. I identified gaps and then went on to prove that there were real live hosts in those gaps. For goodness sake...they're listed on Senderbase, and they also show up in real examples of mail found online.

You can always ssend an additional probe if the configuration has changed.

But the configuration hasn't changed....the mailhost data is faulty...full of holes, and so it can't properly evaluate all mail sent through the hosts for which the data is incomplete.

This system is NOT ready for prime time! It had better NOT become mandatory, and we had better be able to dump out of it if it's not going to be significantly less random. Yahoo, as my example showed, has hundreds of mailhosts, and the system isn't going to discover them all by trial and error.

DT

Edited September 17, 2004 by DavidT

[DavidT 0]

Update: I put through some additional mailhosts probes to try to solve the problem with incomplete data in my mailhosts and ran into the "appears to traverse more than one domain" error, so I wound up having to resolve it by requesting another waiver. I also sent an email to deputies with some additional details and Richard manually added an IP address to one of my hosts, which has helped with the parsing problems.

When I inspect the "Hosts/Domains" lists for some of my mailhosts, however, I'm still seeing the gaps I reported before. I sent a series of test messages to one of my addresses that's handled through the "emaildefenseservice.com" system, and some of them passed through server names that don't show up in my mailhosts list, although the parser is now OK with that.

I think what's needed is an explanation of the significance of those "Hosts/Domains" lists (the dropdown lists for each mailhost) in terms of the issues I've raised. They appear to be quite disorganized, in that they're incomplete and don't appear to be sorted in any obvious order.

Once a user has configured a particular mailhost, do those lists ever get updated? In other words, are they static, or dynamic?

DT

[StevenUnderwood 0]

Once a user has configured a particular mailhost, do those lists ever get updated? In other words, are they static, or dynamic?

I will tell you that they appear dynamic to me.

When I originally configured mailhost, my charter config consisted of about 10 Hosts/Domains: and only a few Relaying IPs:. I know have 40 Hosts/Domains: and 15 Relaying IPs:

When I originally configured mailhost, my yahoo config consisted of about 25 Hosts/Domains: and about 20 Relaying IPs:. I know have nearly 500 (491 by my count) Hosts/Domains: and 125 Relaying IPs:

That is part of the nature of the shared configuration. Anytime a host or IP is associated with a domain, it is added to everyone configured for that domain.

Edited September 17, 2004 by StevenUnderwood

[DavidT 0]

When I originally configured mailhost, my yahoo config consisted of about 25 Hosts/Domains: and about 20 Relaying IPs:.  I know have nearly 500 (491 by my count) Hosts/Domains: and 125 Relaying IPs

Here are some of the current hostnames in the SC Mailhosts Yahoo list:

vmk

yipvmb

yipvmg

Those are supposed to be valid host names, but have no "dot domain dot ext" at the end. The list is flawed.

Anytime a host or IP is associated with a domain, it is added to everyone configured for that domain.

Including mistakes such as those shown above. The system is currently random and haphazard. It needs to be changed.

DT

[StevenUnderwood 0]

Those are supposed to be valid host names

And how do you know there is not a valid host within Yahoo somewhere (even if it were only active for a short time) named vmk that handled email at one time. There is also a host vmk.prodigy.net on that list. Also yipvmb.prodigy.net is in the list.

Probably, prodigy used the internal only name somewhere in their headers during one of the probes, so it was added legitimately.

They may also be internal only host names that are not reachable via the internet. Any host or domain name seen within the returned probe for that domain needs to be inserted into the config so spamcop will see it and match it up during the parse.

Edited September 17, 2004 by StevenUnderwood

[DavidT 0]

This just in from Ellen (via email):

...those that you say are wrong for yahoo happen to be correct because of the noncompliant way that they write their headers. They are not glitches and are in there

intentionally.

OK, thanks for the clarification. So, if Yahoo! was following the rules, those odd truncated names wouldn't be there, but then Yahoo! rarely follows anyone else's rules....they're the 800 pound gorilla that doesn't play well with others.

DT

Edited September 17, 2004 by DavidT

[Ellen 0]

But that's haphazard and random, and not a very thorough way to do things with the bigger email systems.

I didn't say that. I identified gaps and then went on to prove that there were real live hosts in those gaps. For goodness sake...they're listed on Senderbase, and they also show up in real examples of mail found online.

But the configuration hasn't changed....the mailhost data is faulty...full of holes, and so it can't properly evaluate all mail sent through the hosts for which the data is incomplete.

This system is NOT ready for prime time!  It had better NOT become mandatory, and we had better be able to dump out of it if it's not going to be significantly less random. Yahoo, as my example showed, has hundreds of mailhosts, and the system isn't going to discover them all by trial and error.

DT

17290[/snapback]

I am not going to argue with you about this. If you have *specific* parsing problems and have a tracking url then I will look at it. The existence of IPs with rDNS names that fill in the gaps that you perceive does not mean that those IPs and/or domain names ever do or will appear in received headers or need to be in mailhosts. You are theorizing from a specific situation that you encountered (have I even seen the tracking url for your problem, I can't remember) to a generalization that may or may not be valid. Without proof that a problem does indeed exist I see no point in my continuing to discuss this.

I am sorry of you think this is harsh but until I see proof of widespread problems we are just having a theoretical discussion that while perhaps interesting is not something I have time to pursue.

[DavidT 0]

(snip)...does not mean that those IPs and/or domain names ever do or will appear in received headers...(snip)

The "missing" host names do indeed appear in received headers...I've already demonstrated that. But what you've now clarified, both via email and here in the forum, is that it shouldn't matter what hostnames appear on those drop-down lists, as long as the rDNS for the servers is correct, because the parser apparently uses the information obtained from rDNS and not simply the host names currently in the SC system during the parsing.

In other words, the host names on those lists are only a starting point, and are basically meaningless, when it comes to the actual parsing. That really ought to be clarified to those people who configure mailhosts. Had I known that, I wouldn't have wasted a lot of my time (and yours as well) discussing this issue. So I too consider it closed, but not entirely satisfactorily resolved until better information is published about the details of what is displayed on our Mailhosts configuration page.

Thanks, Ellen.

DT

[dbiel 0]

David, I am a bit currious why you ignored Ellen's request to post tracking ULR's, from my reading of her post, nothing will be resolved without them. It would help all of us if you could do so. We can report what we see, but the interpretation is a bit of conjecture so I doubt very much if it would be added to the FAQ.

I agree with you that it is an important issue and should be added to the FAQ, but without confirmation it remains unresolved. So I too request that you post a few examples by posting the Tracking ULR's or forwarding them to Ellen and posting that fact.

Thank you.

Edited September 18, 2004 by dbiel

[DavidT 0]

David, I am a bit currious why you ignored Ellen's request to post tracking ULR's

I sent them early on via email, as per the instructions found in the Pinned item:

Mailhost Issues - please read before posting, Please Read BEFORE Posting

The original parsing issue involved an incomplete configuration of an intermediary host, which was resolved by another deputy, so I don't have any new tracking URLs to post.

My basic complaint about the haphazardness of the mailhosts displayed in our drop-down lists remains valid. I hope it gets addressed at some point, but at least this topic will remain archived here for other people who come along with the same concerns that I have.

DT

[dbiel 0]

I sent them early on via email, as per the instructions found in the Pinned item:

Mailhost Issues - please read before posting, Please Read BEFORE Posting

The original parsing issue involved an incomplete configuration of an intermediary host, which was resolved by another deputy, so I don't have any new tracking URLs to post.

My basic complaint about the haphazardness of the mailhosts displayed in our drop-down lists remains valid. I hope it gets addressed at some point, but at least this topic will remain archived here for other people who come along with the same concerns that I have.

DT

17357[/snapback]

A very good start, but you forgot one important thing, They get so many messages and Ellen may not be the only one receiving and processing them, that she might not have even seen it.

It would be helpful to send another (when it becomes available) and include a reference to you post here.

It is easy to think once sent should be enought (I am guilty of that as well) but when you are handling a vast amount of material it is hard to keep related items together.

Thanks for posting.

Edited September 18, 2004 by dbiel

[Ellen 0]

The "missing" host names do indeed appear in received headers...I've already demonstrated that. But what you've now clarified, both via email and here in the forum, is that it shouldn't matter what hostnames appear on those drop-down lists, as long as the rDNS for the servers is correct, because the parser apparently uses the information obtained from rDNS and not simply the host names currently in the SC system during the parsing.

In other words, the host names on those lists are only a starting point, and are basically meaningless, when it comes to the actual parsing. That really ought to be clarified to those people who configure mailhosts. Had I known that, I wouldn't have wasted a lot of my time (and yours as well) discussing this issue. So I too consider it closed, but not entirely satisfactorily resolved until better information is published about the details of what is displayed on our Mailhosts configuration page.

Thanks, Ellen.

DT

17350[/snapback]

They are not necessarily meaningless.

[DavidT 0]

They are not necessarily meaningless.

A little more elaboration would be appreciated, then. I wrote:

...the host names on those lists are only a starting point, and are basically meaningless, when it comes to the actual parsing...

because I've shown that the lists are haphazard (representing only hosts through with confirmed probes happened to have travesed) and far from complete. In an earlier response, you wrote:

"I am not going to argue with you about this."

and yet your "not necessarily meaningless" comment would appear to be exactly that. I'm hoping that more complete and meaningful information can be brought to light about this issue, so further clarification, rather than simple defense of the system as it currently exists, would be welcome.

DT