Hi Jeroen,
Let's not start this email security discussion by being focused on the bottom line or by being cynical and say some network will not care. We all know that out there, that shouldn’t stop legitimate providers from getting their act together. Let’s keep rational and positive.
Personally, I have enough resources available for protecting my network regardless of a technology or a vendor, so I do not want to take the problem by assuming how much money do I have to put on the table. The thing is, relying on a proprietary (fictional) protection appliance which filters SMTP at will, based on unknown decisions factors is probably not the way to solve issues and is probably not future-proof as the spammers tend to have a few smartass and they have their own ecosystems for acquiring new spam technology. However, if some do work brilliantly, with very little false-negative/false-positive overheads, why not give them a chance.
Getting back to the business, let me describe the goal of my previous message and the current deployments we have to cover such problems. I do not talk about DPI deployment and I am solely trying to solve abused-SMTP usage. The ultimate goal here is to protect IP reputation of servers. I do not want to filter dial-in, DSL, 3G network, but a Hosting network where the SMTP server will accept authentified customers to send their emails as long as they have proper credentials.
Let me describe what I call a spam in this case, because the word spam holds multiple definitions and multiple usages and hardly get everybody to agree on its meaning. To me a spam is a message which is unsolicited, by opposition to a one-to-one, one-to-many or a newsletter communications, which may or may not be wanted by the recipients, but assuming the double opt-in was done correctly, this is different and it requires administrative handling rather than technical handling. In my case, I refer to customers who get their accounts compromised (either POP/IMAP and SMTP) in their email clients such as Outlook, Thunderbird or their Web Account (FTP, HTTP) with either some malicious piece of code on their computers or compromised self-made or well-known un-updated, compromised CMS such as Joomla or home-made poorly programmed scripts.
We have two types of customers. Professional customers who know what they are doing. This kind of customer has no outgoing spam issue. They add a specific tag to their outgoing email, their emails are relayed to a smarthost which strips the header and do a few tricks such as DKIM signing and ultimately relay the message into the Internet. We can assume they send clean emails, unless somebody don’t want to receive them, but we handle this by Feedback Loop and that happens with a ratio of 1:200000. In addition, these customers usually have good security measures in place and we rarely see Trojan-infected hosts within their network. For the latter, we rate-limit the number of email they can send per hour (on some hosts) we look at the type of return we get in the incoming logs and we correlate sudden bursts of outbound message and NDR. Often we can quickly workout compromised accounts and we can identify the script, by looking at the source URI of the script or the SMTP- authenticated account which we reset the password and call the customer or wait for the call. This is usually not a rocket science and it requires a lot babysitting and don’t always prevent the arm of the being RBL.
It is actually way harder to protect regular, simple, customer sending an email with his email client than protecting scripts and mass-mailing customers. Now anybody who dealt with an Outgoing Spam attack have seen that spammers are not always subtle and they just try to flood any address usually within aol, gmail and hotmail with low-quality spam to help solve impotence. Shall we just provide free blue pills to just solve the issue, or are they cheaper ways to get these behaviours to trigger some mechanism that pause/drop and give a chance to resume the mailing or just drop it dead? I am not against a system which is not perfect but which can be influenced by the administrators to not reproduce mistakes.
Maybe somebody has more insights to share at this point. If somebody does, they are welcome most welcome.
Cheers, Gregory
On 25 January 2013 12:08, Jeroen Massar jeroen@massar.ch wrote:
On 2013-01-25 11:47, Gregory Agerba wrote:
Hi everybody,
This is a call to all ISPs and Hosters which operates shared or private environments where outbound spam is an issue.
My goal here is not to collect selfishly as many possibilities for myself, but to collect them and see how they can be tested, implemented and documented to effectively (try to) eradicate spam. If they tend to work better than the one I am currently working on, I will put together a small presentation and share the outcome.
Maybe the best first question is then: what are you working on?
The next question are:
- how much money do you have to burn?
- what type of network do are you looking at? DSL / Cable / GPRS/UMTS/EDGE / etc do devices get static IPs / dynamic IPs / NAT/CGN? do you have logs of which IP is at which customer etc
- how do you keep in contact with your customers/users?
- what are your other constraints?
Everybody wants to shift the spam outside of his network
I think you mean that you want to not want to allow spam to go outside of your network and that you want to catch it before it leaves?
and everybody knows it is easier to filter inbound spam.
Why? Spam is spam, if you can filter it outbound you can filter it inbound. The only thing you need to do for outbound is to force everything through the filter and that is where the nastyness comes.
Instead of filtering you could also have a proper and quick abuse setup, hence the "how do you keep in contact" above.
It is in the interest of all networks and the public to effectively work and implement these solutions.
Unfortunately that is where a major problem lies: not all networks care.
There are lots of networks that do not care at all, they get paid and that is what they care about.
Also, the larger your operation (eg, think the size of a Cablecom/Swisscom) the more costly it is to stop and resolve spam from leaving your network, either because one has to force those customers through a filter/rate-limit setup or because contacting them all is costly.
Note that are residential ISPs who set up walled-garden approaches, thus at first strike mail the customer, second strike call the customer, third strike put them in a walled garden so they get a nice webpage telling them how bad they are.
But depending on the contract they have you might not be able to pull that off, also VoIP can be an issue when you block them that way.
Oh, and when you force them through a SMTP filter it is very nice to actually tell that to your customers, something that for instance Swisscom still has not done and it is still impossible to find that they are do, there is www.swisscom.ch/p25/ but that contains no details whatsoever nor have they actually informed customers that they where changing their internet connectivity to a filtered type.
Let alone wondering what the heck they imply with
"Aside from the spam filter for all Bluewin e-mail accounts, Swisscom has started operating an additional spam filter on the DSL network. This filter now also checks e-mail from free e-mail providers such as GMX, Google Mail and Hotmail if the e-mail is sent from a Swisscom connection."
Does that mean they sniff port 80 and 443 too so they can filter GMX/Gmail/Hotmail or do they just mean port 25 (likely, but not sure)
"As a result of the introduction of the new spam filter, e-mail that is sent with SMTP authentication via port 25 can no longer be sent."
Intercepting IP traffic is such a bad precedence for an ISP (because, then they can also do that to other ports/protocols/IPs etc) and the question of course also still is if that is even LEGAL in Switzerland...
At the end of the research, I will get in touch with every contributor which had a concept, an idea or a working solution and see if and how they want to be credited within the document. To avoid useless noise, I will suggest sending me your suggestions off-list.
It would be nice if you would start with explaining what you already have, as then people can add to that.
Greets, Jeroen
On 2013-01-25 13:06, Gregory Agerba wrote:
Hi Jeroen,
Let's not start this email security discussion by being focused on the bottom line or by being cynical and say some network will not care. We all know that out there, that shouldn’t stop legitimate providers from getting their act together. Let’s keep rational and positive.
But that is the rational thing. One will never fully solve spam because of it. Knowing your limits is a good thing.
Personally, I have enough resources available for protecting my network regardless of a technology or a vendor, so I do not want to take the problem by assuming how much money do I have to put on the table.
The money I meant is also the portion that pays you and more importantly the costs for contacting customers to mediate the problem causing why they where sending spam in the first place...
Support/Helpdesk is what costs money too ;)
The thing is, relying on a proprietary (fictional) protection appliance which filters SMTP at will, based on unknown decisions factors is probably not the way to solve issues and is probably not future-proof as the spammers tend to have a few smartass and they have their own ecosystems for acquiring new spam technology. However, if some do work brilliantly, with very little false-negative/false-positive overheads, why not give them a chance.
First ask which issue you want to solve, do you want to just filter or do you want to fix the customer end? See the list of questions in my mail.
Getting back to the business, let me describe the goal of my previous message and the current deployments we have to cover such problems. I do not talk about DPI deployment and I am solely trying to solve abused-SMTP usage.
Thus the case where an IP has sent spam and you already got an abuse report?
The ultimate goal here is to protect IP reputation of servers.
Which was already, in part, harmed by the above, you already have an abuse report.
I do not want to filter dial-in, DSL, 3G network, but a Hosting network where the SMTP server will accept authentified customers to send their emails as long as they have proper credentials.
Do I understand correctly that you will have:
{ Hosting Network } ---> { SMTP cluster with filtering} -> { Internet }
and do not allow any of the other IPs in the Hosting Network to send outbound SMTP? Depending on the type of customer (see the question in previous mail) they might not want to rely on your SMTP server for instance.
or what is the setup you envision?
Let me describe what I call a spam in this case, because the word spam holds multiple definitions and multiple usages and hardly get everybody to agree on its meaning.
[..]
SPAM / UCE / UBE is very well defined.
See for instance http://en.wikipedia.org/wiki/Spam_(electronic) and various other places.
We have two types of customers.
[... you force customers to use your SMTP server + filter ..]
It is actually way harder to protect regular, simple, customer sending an email with his email client than protecting scripts and mass-mailing customers.
Actually the exact same techniques apply it just depends if you want to invest money (hence the question about that) in if you want to educate them properly or not... (hence the question about how you contact them).
These kind of customers do not fit the above Hosting Network picture.
Greets, Jeroen
First ask which issue you want to solve, do you want to just filter or do you want to fix the customer end? See the list of questions in my mail.
Baiscally, you want to filter and protect yourself. Your are an ISP not a security assessment company. Warn the customer and send them and dos and dont's about emails.
Thus the case where an IP has sent spam and you already got an abuse
report?
Which was already, in part, harmed by the above, you already have an abuse report.
Yes, that's right. How else could it be? That's the whole point of this message ;-)
Do I understand correctly that you will have: { Hosting Network } ---> { SMTP cluster with filtering} -> { Internet }
Yes, that is about right.
and do not allow any of the other IPs in the Hosting Network to send outbound SMTP? Depending on the type of customer (see the question in previous mail) they might not want to rely on your SMTP server for
instance.
Sorry, but who cares? Either they use the setup you offer them, either they have their own SMTP servers and the problems is no longer your technical responsability. If something happens, they are liable for it, period. Out of scope for my request.
SPAM / UCE / UBE is very well defined. See for instance http://en.wikipedia.org/wiki/Spam_(electronic) and various other places.
Yes, now the question is how do you define the border between prospecting and spamming? Not expecting an answer here, only a court ruling would have any value here. I refer to customers who are sending UBE and are not even aware of it!
Actually the exact same techniques apply it just depends if you want to invest money (hence the question about that) in if you want to educate them properly or not... (hence the question about how you contact them).
One must live in the Disneyland® realm to believe that providers to educate their customers. Back to earth please!
Anything to suggest so far or are we just stuck into a loop of ping-pong email replies that will go nowhere?
Cheers, Gregory
On 25 January 2013 13:36, Jeroen Massar jeroen@massar.ch wrote:
On 2013-01-25 13:06, Gregory Agerba wrote:
Hi Jeroen,
Let's not start this email security discussion by being focused on the bottom line or by being cynical and say some network will not care. We all know that out there, that shouldn’t stop legitimate providers from getting their act together. Let’s keep rational and positive.
But that is the rational thing. One will never fully solve spam because of it. Knowing your limits is a good thing.
Personally, I have enough resources available for protecting my network regardless of a technology or a vendor, so I do not want to take the problem by assuming how much money do I have to put on the table.
The money I meant is also the portion that pays you and more importantly the costs for contacting customers to mediate the problem causing why they where sending spam in the first place...
Support/Helpdesk is what costs money too ;)
The thing is, relying on a proprietary (fictional) protection appliance which filters SMTP at will, based on unknown decisions factors is probably not the way to solve issues and is probably not future-proof as the spammers tend to have a few smartass and they have their own ecosystems for acquiring new spam technology. However, if some do work brilliantly, with very little false-negative/false-positive overheads, why not give them a chance.
First ask which issue you want to solve, do you want to just filter or do you want to fix the customer end? See the list of questions in my mail.
Getting back to the business, let me describe the goal of my previous message and the current deployments we have to cover such problems. I do not talk about DPI deployment and I am solely trying to solve abused-SMTP usage.
Thus the case where an IP has sent spam and you already got an abuse report?
The ultimate goal here is to protect IP reputation of servers.
Which was already, in part, harmed by the above, you already have an abuse report.
I do not want to filter dial-in, DSL, 3G network, but a Hosting network where the SMTP server will accept authentified customers to send their emails as long as they have proper credentials.
Do I understand correctly that you will have:
{ Hosting Network } ---> { SMTP cluster with filtering} -> { Internet }
and do not allow any of the other IPs in the Hosting Network to send outbound SMTP? Depending on the type of customer (see the question in previous mail) they might not want to rely on your SMTP server for instance.
or what is the setup you envision?
Let me describe what I call a spam in this case, because the word spam holds multiple definitions and multiple usages and hardly get everybody to agree on its meaning.
[..]
SPAM / UCE / UBE is very well defined.
See for instance http://en.wikipedia.org/wiki/Spam_(electronic) and various other places.
We have two types of customers.
[... you force customers to use your SMTP server + filter ..]
It is actually way harder to protect regular, simple, customer sending an email with his email client than protecting scripts and mass-mailing customers.
Actually the exact same techniques apply it just depends if you want to invest money (hence the question about that) in if you want to educate them properly or not... (hence the question about how you contact them).
These kind of customers do not fit the above Hosting Network picture.
Greets, Jeroen
-
Gregory Agerba -
Jeroen Massar