Fredy, if I remember it right, about a year ago you mentioned that Init7 is deploying the ULL presence?

Regardless of that, the unbundled service is slowly coming to Switzerland, and soon one may expect TR-069, DHCP based service on the last mile copper. Then it's no longer a dialup, but rather a leased line with low SLA :)

----- Original Message ----

From: Fredy Kuenzler kuenzler@init7.net To: swinog@swinog.ch Sent: Thursday, March 5, 2009 5:50:01 PM Subject: Re: [swinog] BGP over xDSL ... is evil

Reza Kordi schrieb:

...

Do you know if anybody offers a VDSL upstream with BGP dynamic routing?

Customer is in ZUG city.

how evil ... serious: BGP over L2TP dial up infrastructure (yes, xDSL over BBCS _is_ dialup) should not be implemented. We got several inquiries in the past years and refused them all. A customer which believes to be in need of BGP for redundancy should be able to afford a proper leased line | fibre service.

F.

---

swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

Fredy,

how evil ... serious: BGP over L2TP dial up infrastructure (yes, xDSL over BBCS _is_ dialup) should not be implemented. We got several inquiries in the past years and refused them all.

I agree and I do not say that it should be if it is the real purpose of the project.

Anyway, who tells this is not to mesh / enhance an existing fiber/BGP network ?

There it could make sense, for example as a DRP for some vital services (DNS, Mail) ..

A customer which believes to be in need of BGP for redundancy should be able to afford a proper leased line | fibre service.

When you deeply think, it's quite funny. Everybody buys fiber from G&C and 2-3 other companies. In case of physical cut it would be interesting to see how many people get hurt and it would be even more interesting when we know how used are technologies such as C/D-WDM.

I mean, you can have 5 upstreams in your IX, at the end the duct in the road is likely to be the same place.

Anyway, mostly probably the customer doesn't have business-critical need if he is willing to do it over DSL.

- Gregory

--- Gregory Agerba IT Operations Manager MIG Investments SA 14, Route des Gouttes d'Or 2008 Neuchâtel Switzerland Phone +41 32 722 86 02

Mobile +41 78 831 22 45 Fax +41 32 722 86 03 Email g.agerba@migfx.com

Home http://www.migfx.com/ Disclaimer: This communication may contain confidential, proprietary or legally privileged information. It is intended only for the person(s) to whom it is addressed. If you are not an intended recipient, you may not use, read, retransmit, disseminate or take any action in reliance upon it. Please notify the sender that you have received it in error and immediately delete the entire communication, including any attachments. MIG Investments SA does not encrypt and cannot ensure the confidentiality or integrity of external e-mail communications and, therefore, cannot be responsible for any unauthorized access, disclosure, use or tampering that may occur during transmission. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. MIG Investments SA accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provide. -----Original Message-----

From: swinog-bounces@lists.swinog.ch on behalf of Fredy Kuenzler Sent: Thu 05.03.2009 17:50 To: swinog@swinog.ch Subject: Re: [swinog] BGP over xDSL ... is evil

Reza Kordi schrieb:

Do you know if anybody offers a VDSL upstream with BGP dynamic routing?

Customer is in ZUG city.

how evil ... serious: BGP over L2TP dial up infrastructure (yes, xDSL over BBCS _is_ dialup) should not be implemented. We got several inquiries in the past years and refused them all. A customer which believes to be in need of BGP for redundancy should be able to afford a proper leased line | fibre service.

F.

_______________________________________________ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

Reza Kordi schrieb:

Thank you all for the cool ideas and even better quotes that I received today.

Of course I understand that some comments were coming from those who are making their profits specially on BGP interconnects.

"If you need BGP buy my expensive LL is the wrong idea pal."

To clarify: I don't consider BGP over xDSL a bad idea because xDSL gives less revenue. It's purely from a technical perspective. BGP over xDSL will flap way more often than any other connectivity. Think of 10000 geeks globally get their ASN and PI space and cannot afford proper connectivty and get it done via xDSL - we will see a massive increase of BGP updates in the global routing table, which requires stronger routing boxes. That's the only reason why we don't sell it, even though we could have made quite some money in the past.

Everybody: please don't offer BGP over DSL polluting the BGP table for CHF 20 or 40 net revenue. There are other redundancy options and backup solutions using xDSL.

The BGP table contains today ~275k prefixes, and is still growing. Common routers like Cisco 7206VXR cannot hold the table for ages anymore, and filtering techniques with all the disadvantages need to be implemented. Remember http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt - in case you haven't experienced the old times of a 70k-BGP table.

F.

Lukas Beeler wrote: [..]

I understand that routers use ASICs and probably faster memory than servers, but i can't really imagine it to be a problem to pop 4GB memory into a router that's connected directly to the internet.

Now, where am i mistaken?

The fact that you then also have to handle 1.000.000 updates... or can you do a Shortest-Path calculation faster than some of the bright minds on this planet? (I heared both J and C are hiring those folks with really good pay :)

See the e2e list and various other "new internet" style lists for a lot of reasons why huge routing tables will one day be an issue, unless we can keep the silicon much faster.

Reza Kordi wrote:

Hi F,

Can you define the "BGP over xDSL will flap way more"

What shall I expect here? Did you ever test this as redundancy scenario for existing BGP environments?

Depends on the stability of the DSL link, which in my case at home is pretty good (thank you Swisscom ;) If your DSL link is not stable though you will loose TCP sessions and thus BGP sessions and flapperdyflap.

But just test it in your lab: take a couple of full feeds from another box that you hook for real onto the Internet (or something which sents a similar amount of updates), then hook up another box to that and just introduce packet loss on that link: Both routers will be trying to resend packets in both directions, doing SP all the time (see above :) and of course when the session finally breaks it needs to send the full table over the link. Not even talking on how much it will affect the upstreams when you are retracting the prefix all the time...

Most DSL is then of course also asymetric (20/1 or so) which will give some nice effects too...

Fortunately one can filter BGP on ASNs and just exclude those annoying ones...

The big question of course is: WHY!? There are other better protocols for hooking up end-sites that do not affect the global routing tables.

Greets, Jeroen

On 05.03.2009, at 19:28, Lukas Beeler wrote:

On Thu, Mar 5, 2009 at 19:08, Fredy Kuenzler kuenzler@init7.net wrote:

...

Remember http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt

• in

I just skimmed through that, and i wonder if it's still current.

There's some talk about requiring about 128MB of memory, and budget concerns of smaller ISPs.

Now, even expensive FB-DIMM memory by vendors like HP and IBM only costs around 360 CHF for 4 GB. And even small two way x86 boxes max out at around 32 - 48 GB. Even if Cisco and Juniper charge 10x as much, that'd still be only 3600 CHF.

I understand that routers use ASICs and probably faster memory than servers, but i can't really imagine it to be a problem to pop 4GB memory into a router that's connected directly to the internet.

Now, where am i mistaken?

When I started using BGP first (1994), we did run BGP on a 2501 router (2 serial ports of 2Mbps + 1 ethernet 10Mbps) which had maybe 32MB of ram and costed 7'000 CHF at the time. This was not good enough, so years later a 7206 was used with 128MB of RAM (this was probably around 1998)

Those routers which costed like 100'000 CHF at that time. Today, you reach the limits on a 7206VXR with like 512MB on a NPE-300 CPU card. You can not stick more than 512MB ram into it. So you need a bigger router CPU which does support more RAM.

Don't forget that more RAM requires hardware which supports it. So its not enough to simply stick a bigger bar into a box. Also most routers are 32bit CPU's so you get into the hard limits of the CPU if you think of 4GB.

but the way bigger problem is the processing speed. Think of the following. In worst case scenario, EVERY SINGLE PACKET's destination IP has to be searched in a routing table of 4GB. RAM is fast but going through 4GB of ram is still not that fast that it would not affect speed if its done on every single packet. Of course there are caching mechanisms who remember last used IP's because they will likely be reused for the next paket etc. But its not as trivial as it might sound. There's more to routing than just holding the routing table in memory. That part is easy and cheap but processing every packet against it in minimum time is what's costly.

Thats probably the reason why on a NPE-300 processor you can not stick more than 512MB of RAM because it would not be fast enough to handle it anyway. So then you go and buy a NPE-G1 or now a NPE-G2 and you end up with a few thousand CHF bill.

Now multiply this with number of ISP's and BGP routers they have and you see the picture. Big ISP's will take care of the core routers, whatsoever as its their core business. But the multihomed customers at the other side of the planet now has to buy a new router just because you added one route more into the table. This is the global effect.

Talking about a small local ISP we helped put together, we used some relatively low-end pc's running OpenBSD+OpenBGPd on flashdist (openbsd read-only on DiskOnModule flash disk). This was about 3 years ago. The machine has a Celeron 2.4Ghz and 512Mb of RAM, and today, getting full routes from two upstreams, the machine is using 181Mb of RAM and itself routing about 300mbit/s of traffic ...

We run a similar network here. Only drawback at the moment is, that openbsd's openospf deamon does not yet support ospfv3, thus no ipv6...

well i wonder really, whats so bad about dsl ? xdsl is not enough specific, ghdsl, sdsl, hdsl .. that are all leased line version of dsl. if that troubles even the SC modem will trouble. i brought my link up .. and it stayed up for years, except as in front of TIX-1 the cables where under water.

using some adsl/vdsl is not so a good idea, as the uplink may be to slow,and the stability of those setup would shure end up in flapping. if the backbone of SC fails CES and even A/V- DSL variants will fail .. So this redundancy is not the right way.

btw, about expensive router, i dont think on the client side is an fullfeed needed. an second hand router will lower the bill and doing the job as well.

i got asked once from a client to have bgp on his cable connection and an ADSL connection .. the only thing he stopped bothering me was to give him an estimated price for the equipment ;-)

using an 2 uplink firewall like the symantec or whatever will do the job better, and an email relay in front on the isp side will send to both ip's. with dyndns services it would even work if both connection are dynamic. positive story, this will lead to loadsharing, negative about that sometimes the way the connection goes is not under controll. Like p2p programms which are open more than one connection to the same destination. but set to failover only will work in any cases.

Roger

Lukas Beeler wrote:

...

Now, even expensive FB-DIMM memory by vendors like HP and IBM only costs around 360 CHF for 4 GB. And even small two way x86 boxes max out at around 32 - 48 GB. Even if Cisco and Juniper charge 10x as much, that'd still be only 3600 CHF.

I understand that routers use ASICs and probably faster memory than servers, but i can't really imagine it to be a problem to pop 4GB memory into a router that's connected directly to the internet.

Now, where am i mistaken?

There still a lot of hardware around which is at the memory expansion limit. And (talking about Cisco) the IOS images don't tend to get smaller... So finally you end up replacing the whole router or NM-engine for some kilo-$ instead of a relative cheap memory upgrade...

I agree with Fredy's concerns about link stability and flapping, especially for residential services. BGP on DSL can although be deployed as backup solution or, if you're close enough to the "BBCS owner", as main link. The major problem you face there is getting a skilled person when you're in trouble... The help-desk guy/girl you get at the phone does usually don't even know how to spell B-G-P and will ask you why you have 4 of them :-)

Daniele

-- This message has been scanned for viruses and dangerous content by MailGate, and is believed to be clean.

---

swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

BGP is the routing protocol of choice in the international world. If your route is on xDSL and flaps, it has an effect to the whole world who uses full routing tables, That's a gazillon of routers out there and in that scenario it doesnt matter if you use a full or partial routing table. this is your choice on the outgoing path but on the incoming path everyone out there has to learn your'e unreachable / reachable over a certain path.

On 06.03.2009, at 00:20, Zorg 421 wrote:

Hey you're all implying BGP = full routing table. BGP with just a single default route on a cheap link may be a good idea. A proven L3 link failure detection mechanism implemented on a wide range of boxes (standardized!)

---

swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

On Thu, 05 Mar 2009 19:08:08 +0100 Fredy Kuenzler kuenzler@init7.net wrote:

Everybody: please don't offer BGP over DSL polluting the BGP table for CHF 20 or 40 net revenue. There are other redundancy options and backup solutions using xDSL.

BGP over SDSL works fine. But you shouldn't run BGP over a BBCS connection aka ADSL, VDSL with PPPoE or DHCP setup.

If you'll try to run bgp (any other routing protocol won't work), you'll get some broken brains, specially if you run more then one LNS

Ueli