Hello list
I am writing on behalf of a colleague who is operating a small hosting business, mainly focused on the setup of the cms and consulting. He is not on the list and asked me to put his words into it. He had the following dispute with an ISP, but I will let him speak (translation via deepl reviewed by me).
---
We run a small hosting business on three managed servers, which we rent from a well-known Swiss ISP and host our customers (SMEs and individuals) there. We have had the misfortune three times that the IP of one of our servers got on the blacklist "UCEprotect" through no fault of our own: http://www.uceprotect.net
In each of these cases a Zurich-based ISP was at fault, who apparently is involved in this blacklist - he didn't want to tell us how exactly, but in the first case he still apologized and he was able to remove the IP from the blacklist at short notice without any problems. Therefore we assume that he has a great influence there. The first times the IP was blacklisted because there was a chaos with a telephone system (short: bounces on non-existent addresses). Last week the IP came back on UCEprotect because a customer had edited his SPF entry incorrectly (he forgot to enter the IP of the server) - a single mail from our customer to a customer of the mentioned Zurich provider was already enough for an entry on the RBL. There was neither a spam dispatch nor a spamtrap; the wrong SPF automatically led to a blacklisting of the whole IP with more than 200 hosting customers, who then of course got mail problems.
It would be normal and justifiable for a mail to be classified as spam because of a wrong SPF record. However, we find it very questionable that a whole IP is "dragged into the abyss" because of this. Especially since we have been fighting against spam for almost 25 years, keeping our servers clean and thus "fighting on the same side", it is all the more irritating to have such obstacles put in the way by this provider. The fact that customers can adjust the DNS entries and thus the SPF record themselves is normal for many providers. A single hosting customer's mistake should not also affect his provider and dozens of other customers.
The methods used for an automatic entry on the blacklist UCEprotect seem at least questionable. I would like to show the provider that he means well, but that it can easily hit the wrong people - and would be grateful for input. After the first case still said "sorry, you've been good to me", there are no more answers to the question whether he really considers these methods to be useful. What do you think can be done here? I don't have time and money for a legal dispute, and blocking any traffic to his IPs to prevent damage to our IPs would probably not be clean either.
---
So, what is your opinion on the behavior of this ISP? Me, Urs, I am with my colleague and I think, it's not acceptable to block a whole IP just while receiving one or a small number of mail without a correct SPF.
Thank you your thoughts, I will collect it and send it to my colleague.
Urs Müller Schweizerische Bundesbahnen SBB Senior Architekt / Product Owner Informatik Operations-Management / CYBER Poststrasse 6 - Ostermundigen, 3000 Bern 65 urs.bf.mueller@sbb.ch / www.sbb.ch
Hello Urs,
From my long term experience with e-mail (I think I got my first internet email address around 1988 where nobody thought of spam yet) I can tell you the follwoing:
Fighting spam is honorable and its good that some people take it serious. However....
Things like SPF etc can help to block routes coming from the wrong IP and they are used technically. Correcting a SPF however should immediately fix the delivery. If thats not the case, then the DNS cache can delay it. Thats just how the technology works. The only annoying guys I run into who you always require special "treatment" are google and microsoft. Luckily most of my business partners stay away from US mailproviders but some have not understood the danger there yet. Now SPF is domain based. So if SPF triggers, then mails from that IP for that domain must be blocked. Nothing else.
If there is a blacklist involved, in your case, then it's clearly a very aggressive one. I have seen some "companies" who blame themselves to be the good guys, playing the cyberpolice and who are deciding for millions of mailservers whats good and whats bad. But they overblock and take the ISP hostage. I once had to go the legal path because of such a stupid overblocking. And our friends at Swisscom where using them at the time so if this happens, you are disconnected from a lot of people from one minute to the next. This just to show you the huge power these "companies" have and how little due diligence there is. You already have problems talking to them to start with.
What I'm trying to say here is that blacklist providers are not perfect by far. They fight their own private wars and they can be abused for cyberwar or revenge attacks as well. So from the view of a mailserver operator, you should choose well whom you trust to decide who can send you emails and who doesn't. As far as UCEProtect goes (which I have never seen before), I see serious legal issues. You don't know whom you are trusting. There's no names, no legal entity, no contact information on their webpage.They hide which is never a good sign. And if you look at their delisting policy, you see immediately what they real goal is. If an ISP asks for delisting, it can take up to 7 days. However if you PAY them, you can be immediately removed. This means, they have a direct financial interest to have as many hosts listed as it generates sales. (its "only" 89 CHF per IP). So they would definitively not be my hero's fighting the nasty spam...
Spam is reality and everybody hates it. However spam should not be fought primary by simply throwing emails away or blocking. It should be fought primary by taking actions against the spammers itself which makes him stop his odd behaviour. Because otherwise they continue to overload the internet and annoy everyone and making e-mail became useless.
I give you a example:
I run a company in Iceland which has a datacenter. For some reason its sales@.. email address got added into a list of mailing addresses. Since a couple of years I got tons of spams in french advertizing stuff from France which I have absolutely no relation with. For example it was promoting to change my home electricity to another provider which is even impossible unless you live in France. Blocking these emails is not really possible on technical means without lots of collateral damage because they came from valid mailservers and valid companies. Unsubscribe buttons usually did work and reduce the volume already quite a bit. Replying to these people with something like this:
Thank you for your query. Given I have never given you an opt-in to send me spam, I would like to remind you that the law on unfair competition act, especially article 23 togeter with article 333 and 34 of the criminal code defines the maximum penalty for spamming as 1'080'000 CHF or up to 3 years in prison. So please think again before sending the next spam.
and also asking through GDPR to show them the opt-in or where they got the email address from, makes these companies quickly remove that email address and think again if it was a good idea to buy this "email addresses for cause X..." list from some shady seller. These measures have reduced the "French" spam I am getting to very very few. So this was much more effective than blindly trusting a 3rd party to just block everything. Sure >90% of the blocked stuff is spam but there might be a few mails which get deleted which are important for your business. And if its in your spam folder on your computer, you can at least find it if needed. If its deleted before even hitting your machine, you are dead and you don't even yet know it.
The technical things help for mailservers going nuts, anonymous viagra spams etc. But these don't stay for long. If an IP doesn't work anymore, these folks have a million other mailservers to try and just move on.
So to answer your question: someone putting an IP on to a blacklist just because of a mismatch on a SPF is definitively wrong because it affects all emails going through that IP address where the SPF is correct.. And if some autmoated blocks are put in place, for god sake, make it clear why what was blocked and how things can be corrected. And dont outsource that decision to 3rd party just because youre too lazy. It could fire back one day and that could ruin your business.
PS: maybe we should start a blacklist of blacklist providers.. ;-)
On 7 Oct 2020, at 17:42, Mueller Urs SBB CFF FFS urs.bf.mueller@sbb.ch wrote:
Hello list
I am writing on behalf of a colleague who is operating a small hosting business, mainly focused on the setup of the cms and consulting. He is not on the list and asked me to put his words into it. He had the following dispute with an ISP, but I will let him speak (translation via deepl reviewed by me).
We run a small hosting business on three managed servers, which we rent from a well-known Swiss ISP and host our customers (SMEs and individuals) there. We have had the misfortune three times that the IP of one of our servers got on the blacklist "UCEprotect" through no fault of our own: http://www.uceprotect.net
In each of these cases a Zurich-based ISP was at fault, who apparently is involved in this blacklist - he didn't want to tell us how exactly, but in the first case he still apologized and he was able to remove the IP from the blacklist at short notice without any problems. Therefore we assume that he has a great influence there. The first times the IP was blacklisted because there was a chaos with a telephone system (short: bounces on non-existent addresses). Last week the IP came back on UCEprotect because a customer had edited his SPF entry incorrectly (he forgot to enter the IP of the server) - a single mail from our customer to a customer of the mentioned Zurich provider was already enough for an entry on the RBL. There was neither a spam dispatch nor a spamtrap; the wrong SPF automatically led to a blacklisting of the whole IP with more than 200 hosting customers, who then of course got mail problems.
It would be normal and justifiable for a mail to be classified as spam because of a wrong SPF record. However, we find it very questionable that a whole IP is "dragged into the abyss" because of this. Especially since we have been fighting against spam for almost 25 years, keeping our servers clean and thus "fighting on the same side", it is all the more irritating to have such obstacles put in the way by this provider. The fact that customers can adjust the DNS entries and thus the SPF record themselves is normal for many providers. A single hosting customer's mistake should not also affect his provider and dozens of other customers.
The methods used for an automatic entry on the blacklist UCEprotect seem at least questionable. I would like to show the provider that he means well, but that it can easily hit the wrong people - and would be grateful for input. After the first case still said "sorry, you've been good to me", there are no more answers to the question whether he really considers these methods to be useful. What do you think can be done here? I don't have time and money for a legal dispute, and blocking any traffic to his IPs to prevent damage to our IPs would probably not be clean either.
So, what is your opinion on the behavior of this ISP? Me, Urs, I am with my colleague and I think, it's not acceptable to block a whole IP just while receiving one or a small number of mail without a correct SPF.
Thank you your thoughts, I will collect it and send it to my colleague.
Urs Müller Schweizerische Bundesbahnen SBB Senior Architekt / Product Owner Informatik Operations-Management / CYBER Poststrasse 6 - Ostermundigen, 3000 Bern 65 urs.bf.mueller@sbb.ch / www.sbb.ch
swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Hello Urs,
My take on your problem is the following: - SPF is bad and breaks mail delivery, don't use it. But, if someone defines SPF records, and they thus declare they want to shoot themselves into their feet, by all means, I encourage to block mails failing SPF, because that's what domain owners who define SPF records ask for. - everyone can setup blocking lists defining the oddest criteria for when an entry gets added. Someone in charge of a mailserver can decide based on his criteria, and his alone, what mails he wants to receive and what mail to reject. He can use whatever resources he wants, including bogus lists such as uceprotect. But, it would be prudent to inform yourself about what exactly you enable before you do so... - now, if someone deploys antispam gateways such as those from Sophos, and decides to just click "enable all block lists" or however that menu looks like, and with this enables lists such as uceprotect, then that person just cut their company off from a lot of valid mail. If he wanted to do that, it's his decision, but usually it helps to teach those admins about the errors of their ways, instead of blaming the list provider. - I personally consider uceprotect to be a rogue list with utopian views on how mail service works. Their unlist policy could be considered extortion, but the really responsible party is whoever enables such a list on their servers.
just my personal views:)
Markus
On 20201008, at 09:14, Markus Wild swinog-list@dudes.ch wrote:
My take on your problem is the following:
- SPF is bad and breaks mail delivery, don't use it. But, if someone defines SPF records, and they thus declare they
want to shoot themselves into their feet, by all means, I encourage to block mails failing SPF, because that's what domain owners who define SPF records ask for.
As one of the original people involved in SPF and having been doing this anti-spam dance for some time....
SPF is only a part of a solution to the battle of spam.
It helps a lot to combat broken setups.
If a setup is broken, they are not worthy of receiving mail in the first place.
Thus, if you hate on SPF, I can only conclude you have shot yourself in the foot a lot with it.
As you note though, your server, your problem on how one it is configured.
If you need help in resolving your problems, don't hesitate to ask me directly or here on list so that others can also benefit.
Greets, Jeroen
(Running the full stack of SPF/DKIM/DMARC,MTA-STS, etc etc and not having mail delivery issues... ;)
Hey Jeroen,
SPF is only a part of a solution to the battle of spam.
SPF isn't suited to combat SPAM at all (including the whole other DKIM etc enchilada), since it's quite trivial for spammers to define these records correctly in throwaway domains. Thus, no reasonable spam filter can honour (in a positive way) the presence of an SPF record, they can only punish the connection if there is an SPF record and the connection is in violation of that record. The really only benefit you could get from SPF is some kind of antispoofing protection, but at least in my experience, that is hardly ever a real problem to begin with.
It helps a lot to combat broken setups.
If a setup is broken, they are not worthy of receiving mail in the first place.
Thus, if you hate on SPF, I can only conclude you have shot yourself in the foot a lot with it.
No, I hate SPF because it breaks basic SMTP relaying, or in more enduser speak: redirected mails. Mail is _NOT_ always delivered directly from origin to target, it is quite frequent, that mails get redirected to 3rd party systems. Some SPF advocates just accept their mails failing because they consider mail redirects to be evil. Fine. To really fix those redirect issues, _all_ possibly relaying servers would have to adopt some kind of sender rewriting scheme, which as far as I recall, can blow up sender email addresses to sizes that will exceed RFC standards in very few iterations. Also, in these cases the relaying server will originate 3rd party mails with its own domain name, possibly turning it into a spam funnel. So, for me, SPF is broken by design, and no amount of additional tinkering around its pitfalls will fix that.
Cheers, Markus
On 20201008, at 15:53, Markus Wild swinog-list@dudes.ch wrote:
Hey Jeroen,
SPF is only a part of a solution to the battle of spam.
SPF isn't suited to combat SPAM at all (including the whole other DKIM etc enchilada), since it's quite trivial for spammers to define these records correctly in throwaway domains.
Notice the word "part" above, there are many ways to combat spam, and most are not 100% effective.
Even the closed gardens of the big tech corporations with their millions is not even, heck they are a large source of spam...
Thus, no reasonable spam filter can honour (in a positive way) the presence of an SPF record, they can only punish the connection if there is an SPF record and the connection is in violation of that record. The really only benefit you could get from SPF is some kind of antispoofing protection, but at least in my experience, that is hardly ever a real problem to begin with
It completely stops 'simple attacks' though: hijack/bot a box and spew crap from a random domain and that solves a lot of phishing attempts also with the real domain (does not solve anything like registering a lookalike domain etc though, or just uninformed users).
That is actually most of the nonsense happening in the world and takes care of every host that is not properly configured. And that is the goal of SPF, and also DKIM and also DMARC.
True spammers (read: advertising) are professional organisations and indeed can setup that infrastructure completely without issue.
Thus nothing really works against that as they have more resources and legal constructions to avoid any persecution.
See also the other messages on the list about Rocket Mail and of course this cool GDPR / DSVGO thing.
It helps a lot to combat broken setups.
If a setup is broken, they are not worthy of receiving mail in the first place.
Thus, if you hate on SPF, I can only conclude you have shot yourself in the foot a lot with it.
No, I hate SPF because it breaks basic SMTP relaying, or in more enduser speak: redirected mails.
Rewrite the mail (e.g SRS-style), DKIM sign it, setup proper SPF and voila, it all works fine. Systems I take care of literally send millions of mails that way without issue. (See also http://lists.swinog.ch/public/swinog/2020-August/007358.html)
SMTP Relaying would involve MXs that authorize eachother to accept mail.
You are simply forwarding/redirecting, and the source is not the anymore where it originally came from nor did they give you permission to do so. As such, rewrite the From as you are not that the original sender.
Greets, Jeroen
Am 2020-10-08 15:53, schrieb Markus Wild:
No, I hate SPF because it breaks basic SMTP relaying, or in more enduser speak: redirected mails. Mail is _NOT_ always delivered directly from origin to target, it is quite frequent, that mails get redirected to 3rd party systems. Some SPF advocates just accept their mails failing because they consider mail redirects to be evil. Fine. To really fix those redirect issues, _all_ possibly relaying servers would have to adopt some kind of sender rewriting scheme, which as far as I recall, can blow up sender email addresses to sizes that will exceed RFC standards in very few iterations. Also, in these cases the relaying server will originate 3rd party mails with its own domain name, possibly turning it into a spam funnel. So, for me, SPF is broken by design, and no amount of additional tinkering around its pitfalls will fix that.
Mail-forwarding creates a host of other problems, thus we discourage it.
If you accept a spam-mail (for whatever reason) and it gets forwarded, the other side may decide that you are the spammer and block your IP.
Arguably, this can be minimized with better ingress spam control (and maybe egress spam control) - but you never know what somebody on the other side may deem to be spam and what not.
The large mail-providers will tighten the screws ever more so slightly, so people will have to learn how to fix their mail (or use a 3rd-party service that send from a subdomain...).
There's a reason that even UBS and Credit-Suisse, who long seemed unable to add SPF records (and still refuse to add DKIM records) now have at least SPF records.
On 20201008, at 17:49, Maxim Samo maxim@swill.org wrote:
Last time I looked ubs.com does use DKIM, SPF, and DMARC.
Easy to look:
There is SPF:
$ dig +short ubs.com txt|grep v=spf "v=spf1 include:spf-a.ubs.com include:spf-hosted.ubs.com include:spf.protection.outlook.com -all"
There is definitely DKIM (grabbed from random message):
8<------- Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ubs.com; h=date :date:content-transfer-encoding:content-type:content-type :mime-version:subject:subject:message-id:from:from:received :received:received:received:received:received; s=srsa2048; t= 1601855040; bh=.... Return-Path: noreply-alerting@ubs.com ------->8
And finally DMARC:
$ dig +short _dmarc.ubs.com txt "v=DMARC1; p=reject; sp=reject; rua=mailto:dmarc-reports-57574806756964364169@open.ch,mailto:dmarc-reports@ubs.com; ruf=mailto:dmarc-forensics@ubs.com; rf=iodef; pct=100; ri=86400"
Seems the fine folks at open.ch have a hand in it ;)
If they did not.... then they would not be able to deliver their mail. And fortunately marketing folks win it over security folks (who can't get other things fixed), thus it get implemented.
Greets, Jeroen
I think Markus misses the main point of the OP. The question of the OP is not whether SPF is a good or bad idea in general.
The question is whether its reasonable to add a blacklisting entry for the server IP if you receive a single mail with a non-matching SPF record (ie, server with IP a.b.c.d not covered by the SPF for foo.com http://foo.com/).
My take on this is that this blacklisting behaviour is entirely unreasonable.
And I agree that it’s questionable why anyone should use a list with such unreasonable listing practices (apart from their extortion practices). Unfortunately the burden is first shifted not onto the admins responsible for such stupid decisions, but on the sending party who must wade through massive layers of ignorance and denial.
— Matthias
Am 08.10.2020 um 09:14 schrieb Markus Wild swinog-list@dudes.ch:
Hello Urs,
My take on your problem is the following:
- SPF is bad and breaks mail delivery, don't use it. But, if someone defines SPF records, and they thus declare they
want to shoot themselves into their feet, by all means, I encourage to block mails failing SPF, because that's what domain owners who define SPF records ask for.
- everyone can setup blocking lists defining the oddest criteria for when an entry gets added. Someone in charge of a
mailserver can decide based on his criteria, and his alone, what mails he wants to receive and what mail to reject. He can use whatever resources he wants, including bogus lists such as uceprotect. But, it would be prudent to inform yourself about what exactly you enable before you do so...
- now, if someone deploys antispam gateways such as those from Sophos, and decides to just click "enable all block
lists" or however that menu looks like, and with this enables lists such as uceprotect, then that person just cut their company off from a lot of valid mail. If he wanted to do that, it's his decision, but usually it helps to teach those admins about the errors of their ways, instead of blaming the list provider.
- I personally consider uceprotect to be a rogue list with utopian views on how mail service works. Their unlist policy
could be considered extortion, but the really responsible party is whoever enables such a list on their servers.
just my personal views:)
Markus
swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Thank you guys
My colleague just wrote me:
"I got the following reply from the swiss provider that obviously works closely with UCEprotect that I'd also call a rogue list as of now. This is the reply after sending them all of your arguments which I thank you for. They do obviosuly not only not understand the basic problem but also seem to be more keen on posting bla bla and accusations rather than arguments that make sense. Here is their text:
SPF wurde etabliert, um Phishingmails und sonstige Formen von Betrug zu unterbinden. Wenn der Eigentümer einer Domain solch einen SPF Record setzt, dann gibt er damit automatisch zu verstehen, daß IPs, die nicht erfasst sind, nicht zum Versand von Mails unter seiner Domain berechtigt sind. Ein Server, der Mails versendet, die er ganz offensichtlich nicht versenden darf, beweist, daß er ein gravierendes Sicherheitproblem hat. Euer Diskussionspartner sollte von daher bei Einlieferungen von Kunden schlicht und ergreifend auf SPF prüfen, und Mail, die er laut SPF nicht versenden DARF, von vorne herein nicht annehmen... Dann werden seine Server auch nicht bei uns geblacklistet. Wenn er technisch dazu nicht in der Lage sein sollte, kann das nicht unser Problem sein, UCEPROTECT Server beherrschen das jedenfalls. Statt Ihre und unsere Zeit zu stehlen, um unsinnige Diskussionen über unsere Policies zu führen, hätte er auch seinen Kunden kontaktieren, und diesen zur Abänderung oder Löschung seines SPF Records auffordern können...
Was er aber macht ist ein typisches Lamerverhalten. Statt die Ursache des Problems zu beheben, versuchen solche Leute das Symptom abzumildern, und ihr eigenes Versagen anderen in die Schuhe zu schieben. Ja und das Gejammer und Geschwurbel über uns in irgendwelchen Foren und auch auf Webseiten geht uns gepflegt dort vorbei, wo die Sonne nicht scheint...
Wir betreiben die UCEPROTECT Listen für unsere weltweit zahlreichen zufriedenen Anwender... Wenn diese unzufrieden mit den Listen wären, würde die Listen niemand nutzen, und somit hätten die Gelisteten keinen Grund zu jammern. Mit ihrem Gejammer überführen sich Gelistete aber selbst als Lügner.... daß dieser Hoster Euch damit wochenlang belästigt, statt das Problem direkt auf seinem System abzustellen, oder mit seinem Kunden zu sprechen, zeigt jedenfalls, wes Geistes Kind er ist... Ich denke, damit ist alles gesagt..."
My (Urs) opinion, rather rough words, I personally would like to know the name of that provider, but it’s not me to blame him.
Regards, Urs
Von: swinog-bounces@lists.swinog.ch swinog-bounces@lists.swinog.ch Im Auftrag von Matthias Leisi Gesendet: Donnerstag, 8. Oktober 2020 22:35 An: Markus Wild swinog-list@dudes.ch Cc: swinog@lists.swinog.ch Betreff: Re: [swinog] Handling of UCE / RBL while minor misconfigurations
I think Markus misses the main point of the OP. The question of the OP is not whether SPF is a good or bad idea in general.
The question is whether its reasonable to add a blacklisting entry for the server IP if you receive a single mail with a non-matching SPF record (ie, server with IP a.b.c.d not covered by the SPF for foo.comhttp://foo.com).
My take on this is that this blacklisting behaviour is entirely unreasonable.
And I agree that it’s questionable why anyone should use a list with such unreasonable listing practices (apart from their extortion practices). Unfortunately the burden is first shifted not onto the admins responsible for such stupid decisions, but on the sending party who must wade through massive layers of ignorance and denial.
— Matthias
Am 08.10.2020 um 09:14 schrieb Markus Wild <swinog-list@dudes.chmailto:swinog-list@dudes.ch>:
Hello Urs,
My take on your problem is the following: - SPF is bad and breaks mail delivery, don't use it. But, if someone defines SPF records, and they thus declare they want to shoot themselves into their feet, by all means, I encourage to block mails failing SPF, because that's what domain owners who define SPF records ask for. - everyone can setup blocking lists defining the oddest criteria for when an entry gets added. Someone in charge of a mailserver can decide based on his criteria, and his alone, what mails he wants to receive and what mail to reject. He can use whatever resources he wants, including bogus lists such as uceprotect. But, it would be prudent to inform yourself about what exactly you enable before you do so... - now, if someone deploys antispam gateways such as those from Sophos, and decides to just click "enable all block lists" or however that menu looks like, and with this enables lists such as uceprotect, then that person just cut their company off from a lot of valid mail. If he wanted to do that, it's his decision, but usually it helps to teach those admins about the errors of their ways, instead of blaming the list provider. - I personally consider uceprotect to be a rogue list with utopian views on how mail service works. Their unlist policy could be considered extortion, but the really responsible party is whoever enables such a list on their servers.
just my personal views:)
Markus
_______________________________________________ swinog mailing list swinog@lists.swinog.chmailto:swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
On 2020-10-27 00:01, Mueller Urs SBB CFF FFS wrote:
Thank you guys
well, technically, they (UCEProtect) are correct.
If an IP sends a mail with the From: header indicating a domain for which an SPF record exists, and the sending IP is not supposed to send it, then it is a misconfiguration and that IP should not be trying to send such a mail at all.
If it is trying to send that, it indicates it allows forwarding of mails From: domains that it should not be transmitting for. It should not be doing that.
As such, listing it after one such event might be a bit quick, but technically indeed, that server is misconfigured and allowing relaying of messages it should not be.
Now of course, such an event can happen easily, e.g. when somebody configures a forwarding rule and no rewriting happens. And in the day and age of SPF/DKIM/DMARC/ARC, it again tells you, that such an event should not happen as the rewrite should have taken place.
Thus in the end UCEprotect is correct: fix the host on the IP that is sending those mails, as it is sending mails it should not.
Now, when that situation is fixed, then UCEprotect should IMHO be able to remove the block entry though.
Greets, Jeroen
--
My colleague just wrote me:
"I got the following reply from the swiss provider that obviously works closely with UCEprotect that I'd also call a rogue list as of now. This is the reply after sending them all of your arguments which I thank you for. They do obviosuly not only not understand the basic problem but also seem to be more keen on posting bla bla and accusations rather than arguments that make sense. Here is their text:
SPF wurde etabliert, um Phishingmails und sonstige Formen von Betrug zu unterbinden. Wenn der Eigentümer einer Domain solch einen SPF Record setzt, dann gibt er damit automatisch zu verstehen, daß IPs, die nicht erfasst sind, nicht zum Versand von Mails unter seiner Domain berechtigt sind. Ein Server, der Mails versendet, die er ganz offensichtlich nicht versenden darf, beweist, daß er ein gravierendes Sicherheitproblem hat. Euer Diskussionspartner sollte von daher bei Einlieferungen von Kunden schlicht und ergreifend auf SPF prüfen, und Mail, die er laut SPF nicht versenden DARF, von vorne herein nicht annehmen... Dann werden seine Server auch nicht bei uns geblacklistet. Wenn er technisch dazu nicht in der Lage sein sollte, kann das nicht unser Problem sein, UCEPROTECT Server beherrschen das jedenfalls. Statt Ihre und unsere Zeit zu stehlen, um unsinnige Diskussionen über unsere Policies zu führen, hätte er auch seinen Kunden kontaktieren, und diesen zur Abänderung oder Löschung seines SPF Records auffordern können...
Was er aber macht ist ein typisches Lamerverhalten. Statt die Ursache des Problems zu beheben, versuchen solche Leute das Symptom abzumildern, und ihr eigenes Versagen anderen in die Schuhe zu schieben. Ja und das Gejammer und Geschwurbel über uns in irgendwelchen Foren und auch auf Webseiten geht uns gepflegt dort vorbei, wo die Sonne nicht scheint...
Wir betreiben die UCEPROTECT Listen für unsere weltweit zahlreichen zufriedenen Anwender... Wenn diese unzufrieden mit den Listen wären, würde die Listen niemand nutzen, und somit hätten die Gelisteten keinen Grund zu jammern. Mit ihrem Gejammer überführen sich Gelistete aber selbst als Lügner.... daß dieser Hoster Euch damit wochenlang belästigt, statt das Problem direkt auf seinem System abzustellen, oder mit seinem Kunden zu sprechen, zeigt jedenfalls, wes Geistes Kind er ist... Ich denke, damit ist alles gesagt..."
My (Urs) opinion, rather rough words, I personally would like to know the name of that provider, but it’s not me to blame him.
Regards, Urs
*Von:*swinog-bounces@lists.swinog.ch swinog-bounces@lists.swinog.ch *Im Auftrag von *Matthias Leisi *Gesendet:* Donnerstag, 8. Oktober 2020 22:35 *An:* Markus Wild swinog-list@dudes.ch *Cc:* swinog@lists.swinog.ch *Betreff:* Re: [swinog] Handling of UCE / RBL while minor misconfigurations
I think Markus misses the main point of the OP. The question of the OP is not whether SPF is a good or bad idea in general.
The question is whether its reasonable to add a blacklisting entry for the server IP if you receive a single mail with a non-matching SPF record (ie, server with IP a.b.c.d not covered by the SPF for foo.com http://foo.com).
My take on this is that this blacklisting behaviour is entirely unreasonable.
And I agree that it’s questionable why anyone should use a list with such unreasonable listing practices (apart from their extortion practices). Unfortunately the burden is first shifted not onto the admins responsible for such stupid decisions, but on the sending party who must wade through massive layers of ignorance and denial.
— Matthias
Am 08.10.2020 um 09:14 schrieb Markus Wild <swinog-list@dudes.ch <mailto:swinog-list@dudes.ch>>: Hello Urs, My take on your problem is the following: - SPF is bad and breaks mail delivery, don't use it. But, if someone defines SPF records, and they thus declare they want to shoot themselves into their feet, by all means, I encourage to block mails failing SPF, because that's what domain owners who define SPF records ask for. - everyone can setup blocking lists defining the oddest criteria for when an entry gets added. Someone in charge of a mailserver can decide based on his criteria, and his alone, what mails he wants to receive and what mail to reject. He can use whatever resources he wants, including bogus lists such as uceprotect. But, it would be prudent to inform yourself about what exactly you enable before you do so... - now, if someone deploys antispam gateways such as those from Sophos, and decides to just click "enable all block lists" or however that menu looks like, and with this enables lists such as uceprotect, then that person just cut their company off from a lot of valid mail. If he wanted to do that, it's his decision, but usually it helps to teach those admins about the errors of their ways, instead of blaming the list provider. - I personally consider uceprotect to be a rogue list with utopian views on how mail service works. Their unlist policy could be considered extortion, but the really responsible party is whoever enables such a list on their servers. just my personal views:) Markus _______________________________________________ swinog mailing list swinog@lists.swinog.ch <mailto:swinog@lists.swinog.ch> http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog <http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog>
swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
well, technically, they (UCEProtect) are correct.
Sure, if I shoehorn my own rules, I can almost guarantee that _I_ will be compliant. That doesn't make them any more relevant though.
If an IP sends a mail with the From: header indicating a domain for which an SPF record exists, and the sending IP is not supposed to send it, then it is a misconfiguration and that IP should not be trying to send such a mail at all.
But SPF is supposed to be applied to the SMTP envelope address, NOT whatever happens to be put into any header lines... For headers you got your other voodoo, like DKIM. And I'm also not so sure that this is a misconfiguration, since that would assume I'd give SPF the same kind of authority as I give basic SMTP relaying, which I don't. I'll repeat myself: SPF is broken by design.
But, I'll actually consider putting in an option into our mail configuration pannels, that lets customers who are forwarding mail to 3rd party servers, reject inbound mails if their forwarding would violate the sender policy. After all, it's our customers who pay us to handle their mail, not external mail senders, and thus customers should have the last word in this decision. Given though how many mail delivery errors never make it back to the sender, I wonder if this added complexity would really be beneficial. My gut feeling is: nope.
The reply of UCEProtect fits my impression I got of them, and it confirms to me that I can't take them seriously. I'll let them throw tantrums in their playpen and move on.
Cheers, Markus
I assume this swiss provider has never allowed any customer to leave.
Every customer who changes DNS providers becuase their marketing decides on a new website and not telling anyone and then their marketing department hijacking the DNS, and forgetting that they run some mailing via some application hosted with you and not caring about spf records becuase the webdesigner doesn't know it, will get you on the blacklist.
that's a scenario which is clearly illegal by this hosting provider.
----- Ursprüngliche Mail ----- Von: "Mueller Urs SBB CFF FFS" urs.bf.mueller@sbb.ch An: "Matthias Leisi" matthias@astrum.ch, "swinog" swinog@lists.swinog.ch Gesendet: Montag, 26. Oktober 2020 23:01:02 Betreff: Re: [swinog] Handling of UCE / RBL while minor misconfigurations
Thank you guys
My colleague just wrote me:
"I got the following reply from the swiss provider that obviously works closely with UCEprotect that I'd also call a rogue list as of now. This is the reply after sending them all of your arguments which I thank you for. They do obviosuly not only not understand the basic problem but also seem to be more keen on posting bla bla and accusations rather than arguments that make sense. Here is their text:
SPF wurde etabliert, um Phishingmails und sonstige Formen von Betrug zu unterbinden. Wenn der Eigentümer einer Domain solch einen SPF Record setzt, dann gibt er damit automatisch zu verstehen, daß IPs, die nicht erfasst sind, nicht zum Versand von Mails unter seiner Domain berechtigt sind. Ein Server, der Mails versendet, die er ganz offensichtlich nicht versenden darf, beweist, daß er ein gravierendes Sicherheitproblem hat. Euer Diskussionspartner sollte von daher bei Einlieferungen von Kunden schlicht und ergreifend auf SPF prüfen, und Mail, die er laut SPF nicht versenden DARF, von vorne herein nicht annehmen... Dann werden seine Server auch nicht bei uns geblacklistet. Wenn er technisch dazu nicht in der Lage sein sollte, kann das nicht unser Problem sein, UCEPROTECT Server beherrschen das jedenfalls. Statt Ihre und unsere Zeit zu stehlen, um unsinnige Diskussionen über unsere Policies zu führen, hätte er auch seinen Kunden kontaktieren, und diesen zur Abänderung oder Löschung seines SPF Records auffordern können...
Was er aber macht ist ein typisches Lamerverhalten. Statt die Ursache des Problems zu beheben, versuchen solche Leute das Symptom abzumildern, und ihr eigenes Versagen anderen in die Schuhe zu schieben. Ja und das Gejammer und Geschwurbel über uns in irgendwelchen Foren und auch auf Webseiten geht uns gepflegt dort vorbei, wo die Sonne nicht scheint...
Wir betreiben die UCEPROTECT Listen für unsere weltweit zahlreichen zufriedenen Anwender... Wenn diese unzufrieden mit den Listen wären, würde die Listen niemand nutzen, und somit hätten die Gelisteten keinen Grund zu jammern. Mit ihrem Gejammer überführen sich Gelistete aber selbst als Lügner.... daß dieser Hoster Euch damit wochenlang belästigt, statt das Problem direkt auf seinem System abzustellen, oder mit seinem Kunden zu sprechen, zeigt jedenfalls, wes Geistes Kind er ist... Ich denke, damit ist alles gesagt..."
My (Urs) opinion, rather rough words, I personally would like to know the name of that provider, but it’s not me to blame him.
Regards, Urs
Von: swinog-bounces@lists.swinog.ch swinog-bounces@lists.swinog.ch Im Auftrag von Matthias Leisi Gesendet: Donnerstag, 8. Oktober 2020 22:35 An: Markus Wild swinog-list@dudes.ch Cc: swinog@lists.swinog.ch Betreff: Re: [swinog] Handling of UCE / RBL while minor misconfigurations
I think Markus misses the main point of the OP. The question of the OP is not whether SPF is a good or bad idea in general.
The question is whether its reasonable to add a blacklisting entry for the server IP if you receive a single mail with a non-matching SPF record (ie, server with IP a.b.c.d not covered by the SPF for [ http://foo.com/ | foo.com ] ).
My take on this is that this blacklisting behaviour is entirely unreasonable.
And I agree that it’s questionable why anyone should use a list with such unreasonable listing practices (apart from their extortion practices). Unfortunately the burden is first shifted not onto the admins responsible for such stupid decisions, but on the sending party who must wade through massive layers of ignorance and denial.
— Matthias
Am 08.10.2020 um 09:14 schrieb Markus Wild < [ mailto:swinog-list@dudes.ch | swinog-list@dudes.ch ] >:
Hello Urs,
My take on your problem is the following: - SPF is bad and breaks mail delivery, don't use it. But, if someone defines SPF records, and they thus declare they want to shoot themselves into their feet, by all means, I encourage to block mails failing SPF, because that's what domain owners who define SPF records ask for. - everyone can setup blocking lists defining the oddest criteria for when an entry gets added. Someone in charge of a mailserver can decide based on his criteria, and his alone, what mails he wants to receive and what mail to reject. He can use whatever resources he wants, including bogus lists such as uceprotect. But, it would be prudent to inform yourself about what exactly you enable before you do so... - now, if someone deploys antispam gateways such as those from Sophos, and decides to just click "enable all block lists" or however that menu looks like, and with this enables lists such as uceprotect, then that person just cut their company off from a lot of valid mail. If he wanted to do that, it's his decision, but usually it helps to teach those admins about the errors of their ways, instead of blaming the list provider. - I personally consider uceprotect to be a rogue list with utopian views on how mail service works. Their unlist policy could be considered extortion, but the really responsible party is whoever enables such a list on their servers.
just my personal views:)
Markus
_______________________________________________ swinog mailing list [ mailto:swinog@lists.swinog.ch | swinog@lists.swinog.ch ] [ http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog | http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog ]
_______________________________________________ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
On 2020-10-27 08:01, Silvan M. Gebhardt wrote:
I assume this swiss provider has never allowed any customer to leave.
Every customer who changes DNS providers becuase their marketing decides on a new website and not telling anyone and then their marketing department hijacking the DNS, and forgetting that they run some mailing via some application hosted with you and not caring about spf records becuase the webdesigner doesn't know it, will get you on the blacklist.
Mail server admin can do a SPF check (or have a list of allowed source email domains) before outbound and reject forwarding these emails.
Of course the marketing/sales department will just "make it work" and "that is your problem"....
Still, technically, something one can stop from wrongly forwarding messages that the host is not supposed to be sending.
Given that one really need properly configured SPF/DKIM/DMARC to have your mail not arrive in Spam folder on the big providers.... it does not make sense to send mail that one cannot send anyway.
Greets, Jeroen
Hi,
On Tue, Oct 27, 2020 at 08:40:39AM +0100, Jeroen Massar wrote:
Mail server admin can do a SPF check (or have a list of allowed source email domains) before outbound and reject forwarding these emails.
I read this and I wonder "which of the MTAs out there can do that" - that is, check SPF (and others) for outgoing mails.
"Blaiming all on the MTA operator" isn't totally reasonable either - you might have a totally valid configuration, and then someone whose mail you legitimately sent before (either forward rules that had no conflicting SPF yet, or your server was listed, or...) changes *their* SPF stuff, making *your* MTA noncompliant.
Is this an error? Yes, surely.
Is the MTA operator to blaim for it? Possibly sometimes, but certainly not "always, and solely".
Gert Doering -- NetMaster
On 2020-10-27 09:04, Gert Doering wrote:
Hi,
On Tue, Oct 27, 2020 at 08:40:39AM +0100, Jeroen Massar wrote:
Mail server admin can do a SPF check (or have a list of allowed source email domains) before outbound and reject forwarding these emails.
I read this and I wonder "which of the MTAs out there can do that" - that is, check SPF (and others) for outgoing mails.
Not many, as normally you do not accept mail at ingress (smtp/maildrop) that you cannot send. I've built custom SMTP engines for people that did this though (and if the domain was not acceptable either reject that or wrap it in a @via.<domain> / SRS-style)
For a ready-off-the-shel-prioject I am sure that Halon (https://halon.io/) can do this though, but for most you would have to custom code a SPF-check-on-exit. (as SPF mostly is applied to inbound).
For everybody else one simply configures a list of domains that one is authoritive for and just use that when mail is dropped in over SMTP or maildrop. Hopefully along with valid DKIM + SPF records outbound and signing etc when it actually goes out of the box.
"Blaiming all on the MTA operator" isn't totally reasonable either - you might have a totally valid configuration, and then someone whose mail you legitimately sent before (either forward rules that had no conflicting SPF yet, or your server was listed, or...) changes *their* SPF stuff, making *your* MTA noncompliant.
Is this an error? Yes, surely.
Is the MTA operator to blaim for it? Possibly sometimes, but certainly not "always, and solely".
I don't think blaming somebody is a useful avenue.
As mentioned, marketing can be an annoying thing. For instance I noticed that one of our domains maxed out the SPF inclusion limit... because people just kept on piling on SPF includes, never actually understanding what it is for and what the limitations are. All that marketing spam (and all legit mail) ended up in SPF:permerr land, thus nicely rejected ;)
The reason for my mail was to elaborate what one could do against becoming listed in restrictive lists like UCEProtect.
Making sure one only egress mail that one is supposed to send (SPF/DKIM/DMARC/ARC) is the only way to do that and would mean being a good citizen on the Internet, which is why lists like UCEProtect exist: if you configure your stuff correctly, you won't end up on them.
That said, I personally avoid strict lists like that, but everybody can pick their own lists, your domain, your selection on what to accept for whatever you want.
Greets, Jeroen
Hi,
On Tue, Oct 27, 2020 at 01:00:59PM +0100, Jeroen Massar wrote:
Making sure one only egress mail that one is supposed to send (SPF/DKIM/DMARC/ARC) is the only way to do that and would mean being a good citizen on the Internet,
Much easier said than done...
which is why lists like UCEProtect exist: if you configure your stuff correctly, you won't end up on them.
You totally miss the "you have a contract with the customer to run their mail for them, so of course you accept the mail, and then they mess up their SPF records in DNS" part.
And then your whole mail server is blocked.
Gert Doering -- NetMaster
On 2020-10-27 13:15, Gert Doering wrote:
Hi,
On Tue, Oct 27, 2020 at 01:00:59PM +0100, Jeroen Massar wrote:
Making sure one only egress mail that one is supposed to send (SPF/DKIM/DMARC/ARC) is the only way to do that and would mean being a good citizen on the Internet,
Much easier said than done...
Of course it is easily said, this is a mailinglist, not a big slide deck or a huge howto how to run a mailserver. Needs quite some experience that ;)
For many folks on this list, SPF came, then DKIM, then DMARC, then ARC and their installations started supporting them one by a time, thus evolution is easy.
Starting from 0, not so much.
But, SwiNOG is here to help. If people have setup questions, we can answer them here, another good citizen on the internet is a win for everybody.
which is why lists like UCEProtect exist: if you configure your stuff correctly, you won't end up on them.
You totally miss the "you have a contract with the customer to run their mail for them, so of course you accept the mail, and then they mess up their SPF records in DNS" part.
And then your whole mail server is blocked.
Yes, what UCEProtect does in 'one fail and you are out' is a bit over aggressive. That is completely out of your control. Rejecting the mail would be good enough indeed, as the collateral damage is too much. (Would be fun if they listed Google + MS MXs though, will quickly stop people using those kind of lists... and considering the amount of spam originating through google, though with valid SPF/DKIM etc... should happen at one point :) -- maybe it a threshold "X mails out Y bad, then block", and a low volume sender then gets blocked quicker than a high volume spammer...
One variant: as the domain needs also DKIM + SPF, and if the customer is not as tech savvy: always take over domain hosting...
And/or monitoring DKIM/SPF records that they are valid for your setup and warning the customer that you stop relaying their messages as their setup is wrong.
Greets, Jeroen
Mail server admin can do a SPF check (or have a list of allowed source email domains) before outbound and reject forwarding these emails.
You can check this before you allow the domain to be sent. However, I don't know of any mail server/MTA that can do this out-of-the-box. And note also:
1. You would have to do this before sending every single email, because DNS records can change. As a mail server provider you have no control over it. 2. Even then it can take a while until you as a sender server notice this, thanks to TTL and DNS propagation. 3. The way this (to us unknown provider) works, you can't technically handle the problem (siehe 1. und 2.)
-
Andreas Fink -
Gert Doering -
Jeroen Massar -
Markus Ritzmann -
Markus Wild -
Matthias Leisi -
Maxim Samo -
Mueller Urs SBB CFF FFS -
rainer＠ultra-secure.de -
Silvan M. Gebhardt