DNSSEC issue with swizzonic DNS servers?

List overview All Threads
Download

newer

older

Reminder Invitation for Domain...

Reach Swizzonic Support outside of...

Benoit Panizzon

27 Dec 2022 27 Dec '22

9:45 a.m.

Hi List

Fancy another DNS issue hunt?

We have DNSSEC validation enabled on our BIND DNS Servers.

We started seeing:

no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:202#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:201#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 81.88.58.219#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 195.110.124.196#53

broken trust chain resolving 'www.numberportability.ch/HTTPS/IN': 2a01:8100:2901::1:183:202#53 broken trust chain resolving 'www.numberportability.ch/AAAA/IN': 2a01:8100:2901::1:183:202#53 client @0x803541d60 X.X.X.X#27325 (www.numberportability.ch): query failed (broken trust chain) for www.numberportability.ch/IN/AAAA at query.c:7724

And of course the query fails, disrupting access some some quite important API.

numberportability.ch. 900 IN SOA dns1.swizzonic.ch. hostmaster.swizzonic.ch. 2022121601 10800 3600 604800 86400

$ dig +dnssec RRSIG www.numberportability.ch @dns1.swizzonic.ch ; <<>> DiG 9.16.33-Debian <<>> +dnssec RRSIG www.numberportability.ch @dns1.swizzonic.ch ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 39132 ;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available

So, from my point of view, the authoritative DNS server thinks, this is a recursive query and refuses to answer with the RRSIG, breaking validation of that record.

Do you get to the same conclusion? Can you resolve this host via any other DNSSEC validating nameserver?

I had no success contacting any technical inclined staff willing to look at the issue since the issue started on 16. December via hostmaster@swizzonic.ch by phone or via support@register.it. So if anyone from Swizzonic is reading here, it would be nice to get a direct contact to further investigate that issue.

Mit freundlichen Grüssen

-Benoît Panizzon-

Show replies by date

Rémy DUCHET

28 Dec 28 Dec

11:09 a.m.

Hi,

No issue from here :

dig www.numberportability.ch +trace

; <<>> DiG 9.16.33-Debian <<>> www.numberportability.ch +trace ;; global options: +cmd . 83292 IN NS f.root-servers.net. . 83292 IN NS a.root-servers.net. . 83292 IN NS h.root-servers.net. . 83292 IN NS j.root-servers.net. . 83292 IN NS i.root-servers.net. . 83292 IN NS g.root-servers.net. . 83292 IN NS d.root-servers.net. . 83292 IN NS l.root-servers.net. . 83292 IN NS k.root-servers.net. . 83292 IN NS b.root-servers.net. . 83292 IN NS e.root-servers.net. . 83292 IN NS c.root-servers.net. . 83292 IN NS m.root-servers.net. . 83292 IN RRSIG NS 8 0 518400 20230110050000 20221228040000 18733 . BDbOstO6sdTqBP2/ER7rX0vjTSJUR/dtnPUOg2zFbt23YhLlSYAegU78 bF5/KLREwricXZMNI6VcGzu+Hn4tYRf/soE/Iy07AagG5WBawRFPdeAS 6XVLsbyDDpSkV/RxJoy8fnAyzGiAV4B4lEpYrDiHdSMAIEn0aU/6CSle sKTsrdSucbaYTosg3bM28lcpPmpXwDWD05wFkLavfmzqut+wzGCI4ge2 AAi3apWMgDs/Ccr9UlpgblvOqMHnvJuX+YCgSyQbzFqMZRaJpHVB3UVC MJJzNgarSHWtj2E4DZMRiXJUHSHZv0FRCrJg7zmDXIahvlUJEF9LfUC9 CkM5Hw== ;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 4 ms

ch. 172800 IN NS f.nic.ch. ch. 172800 IN NS a.nic.ch. ch. 172800 IN NS d.nic.ch. ch. 172800 IN NS b.nic.ch. ch. 172800 IN NS e.nic.ch. ch. 86400 IN DS 10 13 2 0E175543A74D9083EA977BAB2BEE98A771995F80982FB796B2B0B9CC 6413D1A6 ch. 86400 IN RRSIG DS 8 1 86400 20230110050000 20221228040000 18733 . BjNNpFn7hCI2Q6QS6f8m26ZFaAjhaYxcFC6W30h5xguJMN9dneex4L+9 E6bTiawb0q6tCfUkfWDj1QX8NprdxxzpNzDFo+Sksysj6vU28gFSTOl/ H84D8BQTlAWvjrQAuNMzUwNlPz1E0OsDzNpMudfhmLp3m89BNzf+ZTBg 0mSQeW4YEOoxjs86A6yVoLlZrV8msJWfotj2jaLAWaFedLLzk43NrUA1 Y1sf8CzTVma7EqHbpWX3CJrgn7ELv9G5NtFVsmNO5yrHh40fl9KJ+hx7 dlxIjuyj+UjiNgwcMC3CsEzukAopbtuZAyYYE0NLVB3qB/YsN9jEl/AC jCFjzg== ;; Received 724 bytes from 192.112.36.4#53(g.root-servers.net) in 76 ms

numberportability.ch. 3600 IN NS dns1.swizzonic.ch. numberportability.ch. 3600 IN NS dns2.swizzonic.ch. numberportability.ch. 3600 IN DS 10556 13 2 2A50FB3DFA2EFE6F2A80F962EA9DE6CDCA3B5B6F09D3C9D7D972902D 173528F8 numberportability.ch. 3600 IN RRSIG DS 13 2 3600 20230123175307 20221226043002 19537 ch. /JgcDzbIftFZ3vNTx5HdzF2V759lA4Cv2uh84ZWP0p1A4y+xs4aLU2ri rN1NrjW4DsMpKlpghPtIWV/m4j0xdA== ;; Received 277 bytes from 2001:678:3::1#53(e.nic.ch) in 0 ms

www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230105000000 20221215000000 10556 numberportability.ch. FuWo8czeDf/KyCcyYXJF+pYkFJ8HsIX4RrW5a9+fIGqtDUVud7+lxPo9 1oW4H1v69+Mf7rze8SdxAsODJwFUQw== ;; Received 185 bytes from 81.88.58.219#53(dns2.swizzonic.ch) in 8 ms

Also nothing here https://dnsviz.net/d/www.numberportability.ch/dnssec/

Rémy

-----Original Message----- From: Benoit Panizzon via swinog swinog@lists.swinog.ch Sent: Tuesday, 27 December 2022 09:45 To: swinog@swinog.ch Subject: [swinog] DNSSEC issue with swizzonic DNS servers?

Hi List

Fancy another DNS issue hunt?

We have DNSSEC validation enabled on our BIND DNS Servers.

We started seeing:

And of course the query fails, disrupting access some some quite important API.

numberportability.ch. 900 IN SOA dns1.swizzonic.ch. hostmaster.swizzonic.ch. 2022121601 10800 3600 604800 86400

So, from my point of view, the authoritative DNS server thinks, this is a recursive query and refuses to answer with the RRSIG, breaking validation of that record.

Do you get to the same conclusion? Can you resolve this host via any other DNSSEC validating nameserver?

Mit freundlichen Grüssen

-Benoît Panizzon-

-- I m p r o W a r e A G - Leiter Commerce Kunden ______________________________________________________ Zurlindenstrasse 29 Tel +41 61 826 93 00 CH-4133 Pratteln Fax +41 61 826 93 01 Schweiz Web http://www.imp.ch ______________________________________________________ _______________________________________________ swinog mailing list -- swinog@lists.swinog.ch To unsubscribe send an email to swinog-leave@lists.swinog.ch

Markus Binz

29 Dec 29 Dec

2:58 p.m.

No, there was a issue.

Yesterday, they went from:

a@xxxxxx:~ # delv www.numberportabilty.ch ;; resolution failed: ncache nxdomain ; negative response, fully validated ; www.numberportabilty.ch. 900 IN -ANY ;-$NXDOMAIN ; ch. SOA a.nic.ch. dns-operation.switch.ch. 2022122811 900 600 1123200 900 ; ch. RRSIG SOA ... ; ND3E0CGF5OGC08781SOJKFRIOOGBGC7E.ch. RRSIG NSEC3 ... ; ND3E0CGF5OGC08781SOJKFRIOOGBGC7E.ch. NSEC3 1 1 0 - ND3FVIJ6P2HTHSRMNP8BLKRQO4274IF6 NS SOA RRSIG DNSKEY NSEC3PARAM ; QOC41J1IOE62CS3DTUQ3LGS7FCTNAUAM.ch. RRSIG NSEC3 ... ; QOC41J1IOE62CS3DTUQ3LGS7FCTNAUAM.ch. NSEC3 1 1 0 - QOC4OII86T8EDR8QICL731093UTED5MJ NS DS RRSIG ; UB1CIAHS4SJVQLK237TVSNLM741LGBVB.ch. RRSIG NSEC3 ... ; UB1CIAHS4SJVQLK237TVSNLM741LGBVB.ch. NSEC3 1 1 0 - UB1ES2OET0HEP9OTMK4O4BP9RVBE01P6 NS DS RRSIG

a@xxxxxx:~ # delv www.numberportability.ch a ; fully validated www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230105000000 20221215000000 10556 numberportability.ch. FuWo8czeDf/KyCcyYXJF+pYkFJ8HsIX4RrW5a9+fIGqtDUVud7+lxPo9 1oW4H1v69+Mf7rze8SdxAsODJwFUQw==

Now they are in the game of deleting customer domains randomly.

Markus

On 28.12.22 11:09, Rémy DUCHET via swinog wrote:

...

Hi,

No issue from here :

dig www.numberportability.ch +trace

; <<>> DiG 9.16.33-Debian <<>> www.numberportability.ch +trace ;; global options: +cmd . 83292 IN NS f.root-servers.net. . 83292 IN NS a.root-servers.net. . 83292 IN NS h.root-servers.net. . 83292 IN NS j.root-servers.net. . 83292 IN NS i.root-servers.net. . 83292 IN NS g.root-servers.net. . 83292 IN NS d.root-servers.net. . 83292 IN NS l.root-servers.net. . 83292 IN NS k.root-servers.net. . 83292 IN NS b.root-servers.net. . 83292 IN NS e.root-servers.net. . 83292 IN NS c.root-servers.net. . 83292 IN NS m.root-servers.net. . 83292 IN RRSIG NS 8 0 518400 20230110050000 20221228040000 18733 . BDbOstO6sdTqBP2/ER7rX0vjTSJUR/dtnPUOg2zFbt23YhLlSYAegU78 bF5/KLREwricXZMNI6VcGzu+Hn4tYRf/soE/Iy07AagG5WBawRFPdeAS 6XVLsbyDDpSkV/RxJoy8fnAyzGiAV4B4lEpYrDiHdSMAIEn0aU/6CSle sKTsrdSucbaYTosg3bM28lcpPmpXwDWD05wFkLavfmzqut+wzGCI4ge2 AAi3apWMgDs/Ccr9UlpgblvOqMHnvJuX+YCgSyQbzFqMZRaJpHVB3UVC MJJzNgarSHWtj2E4DZMRiXJUHSHZv0FRCrJg7zmDXIahvlUJEF9LfUC9 CkM5Hw== ;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 4 ms

ch. 172800 IN NS f.nic.ch. ch. 172800 IN NS a.nic.ch. ch. 172800 IN NS d.nic.ch. ch. 172800 IN NS b.nic.ch. ch. 172800 IN NS e.nic.ch. ch. 86400 IN DS 10 13 2 0E175543A74D9083EA977BAB2BEE98A771995F80982FB796B2B0B9CC 6413D1A6 ch. 86400 IN RRSIG DS 8 1 86400 20230110050000 20221228040000 18733 . BjNNpFn7hCI2Q6QS6f8m26ZFaAjhaYxcFC6W30h5xguJMN9dneex4L+9 E6bTiawb0q6tCfUkfWDj1QX8NprdxxzpNzDFo+Sksysj6vU28gFSTOl/ H84D8BQTlAWvjrQAuNMzUwNlPz1E0OsDzNpMudfhmLp3m89BNzf+ZTBg 0mSQeW4YEOoxjs86A6yVoLlZrV8msJWfotj2jaLAWaFedLLzk43NrUA1 Y1sf8CzTVma7EqHbpWX3CJrgn7ELv9G5NtFVsmNO5yrHh40fl9KJ+hx7 dlxIjuyj+UjiNgwcMC3CsEzukAopbtuZAyYYE0NLVB3qB/YsN9jEl/AC jCFjzg== ;; Received 724 bytes from 192.112.36.4#53(g.root-servers.net) in 76 ms

numberportability.ch. 3600 IN NS dns1.swizzonic.ch. numberportability.ch. 3600 IN NS dns2.swizzonic.ch. numberportability.ch. 3600 IN DS 10556 13 2 2A50FB3DFA2EFE6F2A80F962EA9DE6CDCA3B5B6F09D3C9D7D972902D 173528F8 numberportability.ch. 3600 IN RRSIG DS 13 2 3600 20230123175307 20221226043002 19537 ch. /JgcDzbIftFZ3vNTx5HdzF2V759lA4Cv2uh84ZWP0p1A4y+xs4aLU2ri rN1NrjW4DsMpKlpghPtIWV/m4j0xdA== ;; Received 277 bytes from 2001:678:3::1#53(e.nic.ch) in 0 ms

www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230105000000 20221215000000 10556 numberportability.ch. FuWo8czeDf/KyCcyYXJF+pYkFJ8HsIX4RrW5a9+fIGqtDUVud7+lxPo9 1oW4H1v69+Mf7rze8SdxAsODJwFUQw== ;; Received 185 bytes from 81.88.58.219#53(dns2.swizzonic.ch) in 8 ms

Also nothing here https://dnsviz.net/d/www.numberportability.ch/dnssec/

Rémy

-----Original Message----- From: Benoit Panizzon via swinog swinog@lists.swinog.ch Sent: Tuesday, 27 December 2022 09:45 To: swinog@swinog.ch Subject: [swinog] DNSSEC issue with swizzonic DNS servers?

Hi List

Fancy another DNS issue hunt?

We have DNSSEC validation enabled on our BIND DNS Servers.

We started seeing:

no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:202#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:201#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 81.88.58.219#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 195.110.124.196#53

broken trust chain resolving 'www.numberportability.ch/HTTPS/IN': 2a01:8100:2901::1:183:202#53 broken trust chain resolving 'www.numberportability.ch/AAAA/IN': 2a01:8100:2901::1:183:202#53 client @0x803541d60 X.X.X.X#27325 (www.numberportability.ch): query failed (broken trust chain) for www.numberportability.ch/IN/AAAA at query.c:7724

And of course the query fails, disrupting access some some quite important API.

numberportability.ch. 900 IN SOA dns1.swizzonic.ch. hostmaster.swizzonic.ch. 2022121601 10800 3600 604800 86400

$ dig +dnssec RRSIG www.numberportability.ch @dns1.swizzonic.ch ; <<>> DiG 9.16.33-Debian <<>> +dnssec RRSIG www.numberportability.ch @dns1.swizzonic.ch ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 39132 ;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available

So, from my point of view, the authoritative DNS server thinks, this is a recursive query and refuses to answer with the RRSIG, breaking validation of that record.

Do you get to the same conclusion? Can you resolve this host via any other DNSSEC validating nameserver?

I had no success contacting any technical inclined staff willing to look at the issue since the issue started on 16. December via hostmaster@swizzonic.ch by phone or via support@register.it. So if anyone from Swizzonic is reading here, it would be nice to get a direct contact to further investigate that issue.

Mit freundlichen Grüssen

-Benoît Panizzon-

swinog mailing list -- swinog@lists.swinog.ch To unsubscribe send an email to swinog-leave@lists.swinog.ch

-- Markus Binz, mbinz@solnet.ch, MB44-RIPE, PGPKEY-ABC5F050 SolNet, Internet Solution Provider

Benoît Panizzon

30 Dec 30 Dec

8:16 a.m.

Hi Markus

Thank for the hint regarding the delv tool.

The issue is back this morning:

$ delv @dns1.swizzonic.ch www.numberportability.ch ;; chase DS servers resolving 'numberportability.ch/DS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving 'ch/NS/IN': 195.110.124.196#53 ;; REFUSED unexpected RCODE resolving 'ch/NS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving './NS/IN': 195.110.124.196#53 ;; REFUSED unexpected RCODE resolving './NS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving 'ch/DS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving 'ch/DS/IN': 195.110.124.196#53 ;; broken trust chain resolving 'numberportability.ch/DNSKEY/IN': 195.110.124.196#53 ;; broken trust chain resolving 'www.numberportability.ch/A/IN': 2a01:8100:2901::1:183:201#53 ;; resolution failed: broken trust chain

Does anyone have a contact to a DNS technician working @ Swizzonic. Preferably with a phone number.

-- Mit freundlichen Grüssen -Benoît Panizzon- @ HomeOffice und normal erreichbar -- I m p r o W a r e A G - Leiter Commerce Kunden ______________________________________________________ Zurlindenstrasse 29 Tel +41 61 826 93 00 CH-4133 Pratteln Fax +41 61 826 93 01 Schweiz Web http://www.imp.ch ______________________________________________________

Markus Binz

10:55 a.m.

Well,

you have to point at resolvers:

delv @8.8.8.8 www.numberportability.ch ; fully validated www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230112000000 20221222000000 10556 numberportability.ch. zq+tng6WB4PhUrA3SbyzcAsUy5ObgmMg+L12abuUb5A0s0qz7EzuDa2a wM6qZKU2TJG672mGOQB1OxI5/+JTeg==

delv @9.9.9.9 www.numberportability.ch ; fully validated www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230112000000 20221222000000 10556 numberportability.ch. zq+tng6WB4PhUrA3SbyzcAsUy5ObgmMg+L12abuUb5A0s0qz7EzuDa2a wM6qZKU2TJG672mGOQB1OxI5/+JTeg==

the name server from swizzonic is not supposed to provide you with a answer to all the queries.

Markus

On 30.12.22 08:16, Benoît Panizzon wrote:

...

Hi Markus

Thank for the hint regarding the delv tool.

The issue is back this morning:

$ delv @dns1.swizzonic.ch www.numberportability.ch ;; chase DS servers resolving 'numberportability.ch/DS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving 'ch/NS/IN': 195.110.124.196#53 ;; REFUSED unexpected RCODE resolving 'ch/NS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving './NS/IN': 195.110.124.196#53 ;; REFUSED unexpected RCODE resolving './NS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving 'ch/DS/IN': 2a01:8100:2901::1:183:201#53 ;; REFUSED unexpected RCODE resolving 'ch/DS/IN': 195.110.124.196#53 ;; broken trust chain resolving 'numberportability.ch/DNSKEY/IN': 195.110.124.196#53 ;; broken trust chain resolving 'www.numberportability.ch/A/IN': 2a01:8100:2901::1:183:201#53 ;; resolution failed: broken trust chain

Does anyone have a contact to a DNS technician working @ Swizzonic. Preferably with a phone number.

-- Markus Binz, mbinz@solnet.ch, MB44-RIPE, PGPKEY-ABC5F050 SolNet, Internet Solution Provider Phone: +41 58 517 6223

Benoît Panizzon

11:21 a.m.

Hi Markus

...

the name server from swizzonic is not supposed to provide you with a answer to all the queries.

I guess if I point to our recursive validating caching NS and it does not possess this data in it's cache, it will start by following from the root by asking for _.numberportability.ch to avoid revealing which host it is exactly looking for until it reaches the authoritative DNS for that zone and then ask this one directly for the desired RR.

I guess this is where something is breaking the chain.

I also don't see why the swizzonic DNS which is the authoritative primary should not answer to all queries. Well of course the DNSSEC chain (Signed DS entries) has to be followed from the root over ch. to swizzonic. But everything else should be obtainable from the authoritative server for that zone, right?

Right now, all needed RR within numberportability.ch resolve ok. So maybe the now found and fixed he issue.

rainer＠ultra-secure.de

2:17 p.m.

Am 2022-12-30 11:21, schrieb Benoît Panizzon via swinog:

...

Hi Markus

...
the name server from swizzonic is not supposed to provide you with a answer to all the queries.

I guess if I point to our recursive validating caching NS and it does not possess this data in it's cache, it will start by following from the root by asking for _.numberportability.ch to avoid revealing which host it is exactly looking for until it reaches the authoritative DNS for that zone and then ask this one directly for the desired RR.

I guess this is where something is breaking the chain.

I also don't see why the swizzonic DNS which is the authoritative primary should not answer to all queries.

If I want to or need to ask the (supposedly) authoritative server(s) about a domain, I add +norecurs.

I believe, if you disable recursive queries on the authoritative-server, it will not answer them, even if it technically could.

Does DNSSEC change that?

Fabian Wenk

28 Dec 28 Dec

11:49 a.m.

Hello

On 27.12.2022 09:45, Benoit Panizzon via swinog wrote:

...

Hi List

Fancy another DNS issue hunt?

We have DNSSEC validation enabled on our BIND DNS Servers.

Same for my private servers.

...

We started seeing:

no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:202#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:201#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 81.88.58.219#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 195.110.124.196#53

broken trust chain resolving 'www.numberportability.ch/HTTPS/IN': 2a01:8100:2901::1:183:202#53 broken trust chain resolving 'www.numberportability.ch/AAAA/IN': 2a01:8100:2901::1:183:202#53 client @0x803541d60 X.X.X.X#27325 (www.numberportability.ch): query failed (broken trust chain) for www.numberportability.ch/IN/AAAA at query.c:7724

It all looks fine so far from my end, or did I miss something important?

fabian@flashback:~ % dig -t ns numberportability.ch +dnssec

; <<>> DiG 9.10.6 <<>> -t ns numberportability.ch +dnssec ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28854 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 1232 ;; QUESTION SECTION: ;numberportability.ch. IN NS

;; ANSWER SECTION: numberportability.ch. 900 IN NS dns2.swizzonic.ch. numberportability.ch. 900 IN NS dns1.swizzonic.ch. numberportability.ch. 900 IN RRSIG NS 13 2 900 20230105000000 20221215000000 10556 numberportability.ch. YDc8MgSRBZDVlRBaP5RfxeGZdkYvNkci8N2rpxQ5NsvjWz9M/HDasP6P AAk4H2tJsJyVK0HqghSCuwuTub1opA==

;; Query time: 42 msec ;; SERVER: 2001:8a8:1005:1::2#53(2001:8a8:1005:1::2) ;; WHEN: Wed Dec 28 11:24:10 CET 2022 ;; MSG SIZE rcvd: 215

fabian@flashback:~ % dig www.numberportability.ch +dnssec @dns1.swizzonic.ch.

; <<>> DiG 9.10.6 <<>> www.numberportability.ch +dnssec @dns1.swizzonic.ch. ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 669 ;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 1680 ;; QUESTION SECTION: ;www.numberportability.ch. IN A

;; ANSWER SECTION: www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230105000000 20221215000000 10556 numberportability.ch. 5PpTJZ19GmcEyD8i3iUBWoZdGYECB3Hvdx2JclKfDVKl3KVbuBekf6RL kP1HRSYPhJZak25YeyhKe1oPemHXrw==

;; Query time: 21 msec ;; SERVER: 2a01:8100:2901::1:183:201#53(2a01:8100:2901::1:183:201) ;; WHEN: Wed Dec 28 11:24:22 CET 2022 ;; MSG SIZE rcvd: 185

fabian@flashback:~ % dig www.numberportability.ch +dnssec @dns2.swizzonic.ch.

; <<>> DiG 9.10.6 <<>> www.numberportability.ch +dnssec @dns2.swizzonic.ch. ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14397 ;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 1680 ;; QUESTION SECTION: ;www.numberportability.ch. IN A

;; ANSWER SECTION: www.numberportability.ch. 900 IN A 164.128.159.204 www.numberportability.ch. 900 IN RRSIG A 13 3 900 20230105000000 20221215000000 10556 numberportability.ch. FuWo8czeDf/KyCcyYXJF+pYkFJ8HsIX4RrW5a9+fIGqtDUVud7+lxPo9 1oW4H1v69+Mf7rze8SdxAsODJwFUQw==

;; Query time: 36 msec ;; SERVER: 2a01:8100:2901::1:183:202#53(2a01:8100:2901::1:183:202) ;; WHEN: Wed Dec 28 11:24:31 CET 2022 ;; MSG SIZE rcvd: 185

fabian@flashback:~ %

Also checking at DNSViz it looks fine: https://dnsviz.net/d/numberportability.ch/dnssec/

So either they fixed it in the meantime or then your server may have some issue or something bad in cache.

Best regards, Fabian

Sebastian Philipp

4:25 p.m.

Hi Benoit

So, there is an A record for www.numberportability.ch, and it's signed and resolves and validates without issue for me.

However, when I attempt to look up the AAAA record (or any other RRtype except A), I get the following response from Swizzonic's nameserver:

...

; <<>> DiG 9.18.9 <<>> www.numberportability.ch aaaa @2a01:8100:2901::1:183:201 +dnssec ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44515 ;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1 ;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 1680 ;; QUESTION SECTION: ;www.numberportability.ch. IN AAAA

;; AUTHORITY SECTION: numberportability.ch. 900 IN SOA dns1.swizzonic.ch. hostmaster.swizzonic.ch. 2022121601 10800 3600 604800 86400

numberportability.ch. 900 IN RRSIG SOA 13 2 900 20230105000000 20221215000000 10556

numberportability.ch. SzRBpQzLj0tEmzfg0LN6vBVd6pDYVY5RhaJd8BFKX57yaU1xCEeVFQiB ogAb0xMsVcUMEew15KbjxDyLBGhvsw==

numberportability.ch. 86400 IN NSEC numberportability.ch. A NS SOA MX TXT RRSIG NSEC DNSKEY

numberportability.ch. 86400 IN RRSIG NSEC 13 2 86400 20230105000000 20221215000000 10556 numberportability.ch. nwLoV6Gr+DLINpw+1wARJkj6VCUEIPT3ciZGrmltkBXu7tlW3L9GF0Ht 5kCZbDooM8yMGOow0gI/EdIzYwKA+A==

;; Query time: 26 msec ;; SERVER: 2a01:8100:2901::1:183:201#53(2a01:8100:2901::1:183:201) (UDP) ;; WHEN: Wed Dec 28 16:13:41 CET 2022 ;; MSG SIZE rcvd: 390

Note the response status:

...

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44515

It is a NOERROR rather than NXDOMAIN. This means the name server indicates that the absence of an AAAA record in the response is a NoData [rfc2308] error rather than a NXDOMAIN error, or, in other words, it claims that the domain www.numberportability.ch. exists, but doesn't have an AAAA record.

Now let's turn our eyes to the NSEC record in the response:

...

numberportability.ch. NSEC numberportability.ch. [... some rrtypes]

Here, Swizzonic's nameserver claims that there is no domain between numberportability.ch. and numberportability.ch., i.e. that it does not have any subdomains at all. This is in contrast to the NoData response above, and thus the DNSSEC validator considers the response bogus.

So it appears there is to be some kind of misconfiguration on Swizzonic's side.

Hope this helps in narrowing down the issue.

Regards Sebastian

[rfc2308]: https://www.rfc-editor.org/rfc/rfc2308#section-2.2

Daniel Stirnimann

6 Jan 6 Jan

9:43 a.m.

Hi Benoit

Not sure what the original problem was on the 27th of Dec but the current problem is as follow:

numberportability.ch has an NSEC negative proof at the zone apex which states that there are no other hostnames then numberportability.ch itself.

dig @dns1.swizzonic.ch numberportability.ch. AAAA +dnssec +norec ... ;; AUTHORITY SECTION: numberportability.ch. 900 IN SOA dns1.swizzonic.ch. hostmaster.swizzonic.ch. 2022121601 10800 3600 604800 86400 numberportability.ch. 900 IN RRSIG SOA 13 2 900 20230119000000 20221229000000 10556 numberportability.ch. TyEySTihvSFvdHr+AIOwYV7P/7OwnEkkKmviAfpDM7Ls/7oSkE0YWpKT rtn2OLAcGayrejP3hYYdU9cH7+DddQ== numberportability.ch. 86400 IN NSEC numberportability.ch. A NS SOA MX TXT RRSIG NSEC DNSKEY numberportability.ch. 86400 IN RRSIG NSEC 13 2 86400 20230119000000 20221229000000 10556 numberportability.ch. Cv2K3pWOJ739PgraeAseqUCXegIGJsrN5zmFRa2hKpohwKY/NCSx2RuJ q1PdHXPh6w9Es+Y6btCZNtuRfQ7iZg==

See the NSEC proof in the above query.

A DNSSEC validating resolver which supports and enables synthesized answers from cached NSEC, NSEC3 (rfc8198) will answer follow up queries for this domain name which fall outside the NSEC chain directly with NXDOMAIN. This problem only occurs if there is already a cached NSEC record. I guess this is not unlikely as most web browser do HTTPS and AAAA qtype lookups in paralell to A queries. HTTPS and AAAA do both not exist for numberportability.ch.

Such a DNSSEC validating resolver which synthesizes answers from cached NSEC, NSEC3 records will not log a DNSSEC validation error. The problem is that the NSEC proof is lying and not that it its DNSSEC signature is invalid.

All current open source DNS resolver software support synthesized answers from cached NSEC, NSEC3 (rfc8198). I tested knot-resolver, powerdns-recursor and BIND and unbound. In BIND the configuration option is called "synth-from-dnssec" which is enabled by default since BIND 9.18. In knot-resolver there is no configuration option, it is always enabled. For powerdns-recursor you need v4.5 where it is enabled by default, option "aggressive-nsec-cache-size". For unbound you need v1.17 but it is disabled by default. The option is "aggressive-nsec". Google Public DNS also supports it.

Note, synthesized answers from cached NSEC, NSEC3 is a very useful feature. To quote the unbound documentation [1]:

"Aggressive NSEC can result in a reduction of traffic on all levels of the DNS hierarchy but it will be most noticeable at the root, as typically more than half of all responses are NXDOMAIN.

Another benefit of a wide deployment of aggressive NSEC is the incentive to DNSSEC sign your zone. If you don’t want to have a large amount of queries for non-existing records at your name server, signing your zone will prevent this."

[1] https://unbound.docs.nlnetlabs.nl/en/latest/topics/privacy/aggressive-nsec.h...

It is also very effective against random subdomain attacks, very common attack vector at the moment (See also rfc8198) where rate-limiting queries does not help. A DNS-OARC talk from 2017 by Petr Špaček also compared it to running the root zone locally, https://indico.dns-oarc.net/event/27/contributions/473/attachments/430/725/D....

If you want to trigger the problem on your DNS resolver, you need to query for a NoData answer first e.g.:

dig numberportability.ch. AAAA

The resolver caches the NSEC proof. You can then query for an existing name which will be synthesized because of the "lying" NSEC proof. e.g.:

dig www.numberportability.ch. A -> synthesized NXDOMAIN instead of the answer record

If you are the zone owner of numberportability.ch, you need to tell Swizzonic that they should execute:

pdnsutil rectify-zone numberportability.ch

This will fix the problem temporarily until the zone is changed again by some users.

A possible (note I'm guessing) root problem is that Swizzonic uses a WebFrontend which directly access the Database with SQL statements. This breaks DNSSEC in PowerDNS. One has to use a WebFrontend which uses the PowerDNS API. See also https://github.com/PowerDNS/pdns/wiki/WebFrontends

Cheers, Daniel

On 27.12.22 09:45, Benoit Panizzon via swinog wrote:

...

Hi List

Fancy another DNS issue hunt?

We have DNSSEC validation enabled on our BIND DNS Servers.

We started seeing:

no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:202#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 2a01:8100:2901::1:183:201#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 81.88.58.219#53 no valid RRSIG resolving 'www.numberportability.ch/DS/IN': 195.110.124.196#53

broken trust chain resolving 'www.numberportability.ch/HTTPS/IN': 2a01:8100:2901::1:183:202#53 broken trust chain resolving 'www.numberportability.ch/AAAA/IN': 2a01:8100:2901::1:183:202#53 client @0x803541d60 X.X.X.X#27325 (www.numberportability.ch): query failed (broken trust chain) for www.numberportability.ch/IN/AAAA at query.c:7724

And of course the query fails, disrupting access some some quite important API.

numberportability.ch. 900 IN SOA dns1.swizzonic.ch. hostmaster.swizzonic.ch. 2022121601 10800 3600 604800 86400

$ dig +dnssec RRSIG www.numberportability.ch @dns1.swizzonic.ch ; <<>> DiG 9.16.33-Debian <<>> +dnssec RRSIG www.numberportability.ch @dns1.swizzonic.ch ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 39132 ;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available

So, from my point of view, the authoritative DNS server thinks, this is a recursive query and refuses to answer with the RRSIG, breaking validation of that record.

Do you get to the same conclusion? Can you resolve this host via any other DNSSEC validating nameserver?

I had no success contacting any technical inclined staff willing to look at the issue since the issue started on 16. December via hostmaster@swizzonic.ch by phone or via support@register.it. So if anyone from Swizzonic is reading here, it would be nice to get a direct contact to further investigate that issue.

Mit freundlichen Grüssen

-Benoît Panizzon-

715

Age (days ago)

725

Last active (days ago)

swinog@lists.swinog.ch

9 comments

8 participants

tags (0)

participants (8)

Benoit Panizzon
Benoît Panizzon
Daniel Stirnimann
Fabian Wenk
Markus Binz
rainer＠ultra-secure.de
Rémy DUCHET
Sebastian Philipp