Fredy, if I remember it right, about a year ago you mentioned that Init7 is deploying the ULL presence?

Regardless of that, the unbundled service is slowly coming to Switzerland, and soon one may expect TR-069, DHCP based service on the last mile copper. Then it's no longer a dialup, but rather a leased line with low SLA :)

----- Original Message ----

From: Fredy Kuenzler kuenzler@init7.net To: swinog@swinog.ch Sent: Thursday, March 5, 2009 5:50:01 PM Subject: Re: [swinog] BGP over xDSL ... is evil

Reza Kordi schrieb:

...

Do you know if anybody offers a VDSL upstream with BGP dynamic routing?

Customer is in ZUG city.

how evil ... serious: BGP over L2TP dial up infrastructure (yes, xDSL over BBCS _is_ dialup) should not be implemented. We got several inquiries in the past years and refused them all. A customer which believes to be in need of BGP for redundancy should be able to afford a proper leased line | fibre service.

F.

---

swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

Thank you all for the cool ideas and even better quotes that I received today.

Of course I understand that some comments were coming from those who are making their profits specially on BGP interconnects.

"If you need BGP buy my expensive LL is the wrong idea pal."

-----Original Message----- From: swinog-bounces@lists.swinog.ch [mailto:swinog-bounces@lists.swinog.ch] On Behalf Of Fredy Kuenzler Sent: Donnerstag, 5. März 2009 17:50 To: swinog@swinog.ch Subject: Re: [swinog] BGP over xDSL ... is evil

Reza Kordi schrieb:

Do you know if anybody offers a VDSL upstream with BGP dynamic routing?

Customer is in ZUG city.

how evil ... serious: BGP over L2TP dial up infrastructure (yes, xDSL over BBCS _is_ dialup) should not be implemented. We got several inquiries in the past years and refused them all. A customer which believes to be in need of BGP for redundancy should be able to afford a proper leased line | fibre service.

F.

_______________________________________________ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

Reza Kordi schrieb:

Thank you all for the cool ideas and even better quotes that I received today.

Of course I understand that some comments were coming from those who are making their profits specially on BGP interconnects.

"If you need BGP buy my expensive LL is the wrong idea pal."

To clarify: I don't consider BGP over xDSL a bad idea because xDSL gives less revenue. It's purely from a technical perspective. BGP over xDSL will flap way more often than any other connectivity. Think of 10000 geeks globally get their ASN and PI space and cannot afford proper connectivty and get it done via xDSL - we will see a massive increase of BGP updates in the global routing table, which requires stronger routing boxes. That's the only reason why we don't sell it, even though we could have made quite some money in the past.

Everybody: please don't offer BGP over DSL polluting the BGP table for CHF 20 or 40 net revenue. There are other redundancy options and backup solutions using xDSL.

The BGP table contains today ~275k prefixes, and is still growing. Common routers like Cisco 7206VXR cannot hold the table for ages anymore, and filtering techniques with all the disadvantages need to be implemented. Remember http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt - in case you haven't experienced the old times of a 70k-BGP table.

F.

Hi F,

Can you define the "BGP over xDSL will flap way more"

What shall I expect here? Did you ever test this as redundancy scenario for existing BGP environments?

Cheers, Reza

-----Original Message----- From: swinog-bounces@lists.swinog.ch [mailto:swinog-bounces@lists.swinog.ch] On Behalf Of Fredy Kuenzler Sent: Donnerstag, 5. März 2009 19:08 To: swinog@swinog.ch Subject: Re: [swinog] BGP over xDSL ... is evil? says who?

Reza Kordi schrieb:

Thank you all for the cool ideas and even better quotes that I received today.

Of course I understand that some comments were coming from those who are making their profits specially on BGP interconnects.

"If you need BGP buy my expensive LL is the wrong idea pal."

To clarify: I don't consider BGP over xDSL a bad idea because xDSL gives less revenue. It's purely from a technical perspective. BGP over xDSL will flap way more often than any other connectivity. Think of 10000 geeks globally get their ASN and PI space and cannot afford proper connectivty and get it done via xDSL - we will see a massive increase of BGP updates in the global routing table, which requires stronger routing boxes. That's the only reason why we don't sell it, even though we could have made quite some money in the past.

Everybody: please don't offer BGP over DSL polluting the BGP table for CHF 20 or 40 net revenue. There are other redundancy options and backup solutions using xDSL.

The BGP table contains today ~275k prefixes, and is still growing. Common routers like Cisco 7206VXR cannot hold the table for ages anymore, and filtering techniques with all the disadvantages need to be implemented. Remember http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt - in case you haven't experienced the old times of a 70k-BGP table.

F.

_______________________________________________ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

Lukas Beeler schrieb:

On Thu, Mar 5, 2009 at 19:08, Fredy Kuenzler kuenzler@init7.net wrote:

...

Remember http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt

• in

I just skimmed through that, and i wonder if it's still current.

Yes it is. Of course BOGONs are outdated, but the concept is still valid.

There's some talk about requiring about 128MB of memory, and budget concerns of smaller ISPs.

Now, even expensive FB-DIMM memory by vendors like HP and IBM only costs around 360 CHF for 4 GB. And even small two way x86 boxes max out at around 32 - 48 GB. Even if Cisco and Juniper charge 10x as much, that'd still be only 3600 CHF.

I understand that routers use ASICs and probably faster memory than servers, but i can't really imagine it to be a problem to pop 4GB memory into a router that's connected directly to the internet.

Now, where am i mistaken?

You assume that all gear can actually handle the memory, but a Cisco 3640 can only address 128MB, a Cisco7206VXR-NPE300 can address 256M - both considered as BGP routers for smaller networks until just recently, and they still run in many smaller networks.

Even a state-of-the-art Foundry MLX-4 is not able to handle more than 25 fullfeeds (either up or downstream) for memory limitations.

F.

On 05.03.2009, at 19:28, Lukas Beeler wrote:

On Thu, Mar 5, 2009 at 19:08, Fredy Kuenzler kuenzler@init7.net wrote:

...

Remember http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt

• in

I just skimmed through that, and i wonder if it's still current.

There's some talk about requiring about 128MB of memory, and budget concerns of smaller ISPs.

Now, even expensive FB-DIMM memory by vendors like HP and IBM only costs around 360 CHF for 4 GB. And even small two way x86 boxes max out at around 32 - 48 GB. Even if Cisco and Juniper charge 10x as much, that'd still be only 3600 CHF.

I understand that routers use ASICs and probably faster memory than servers, but i can't really imagine it to be a problem to pop 4GB memory into a router that's connected directly to the internet.

Now, where am i mistaken?

When I started using BGP first (1994), we did run BGP on a 2501 router (2 serial ports of 2Mbps + 1 ethernet 10Mbps) which had maybe 32MB of ram and costed 7'000 CHF at the time. This was not good enough, so years later a 7206 was used with 128MB of RAM (this was probably around 1998)

Those routers which costed like 100'000 CHF at that time. Today, you reach the limits on a 7206VXR with like 512MB on a NPE-300 CPU card. You can not stick more than 512MB ram into it. So you need a bigger router CPU which does support more RAM.

Don't forget that more RAM requires hardware which supports it. So its not enough to simply stick a bigger bar into a box. Also most routers are 32bit CPU's so you get into the hard limits of the CPU if you think of 4GB.

but the way bigger problem is the processing speed. Think of the following. In worst case scenario, EVERY SINGLE PACKET's destination IP has to be searched in a routing table of 4GB. RAM is fast but going through 4GB of ram is still not that fast that it would not affect speed if its done on every single packet. Of course there are caching mechanisms who remember last used IP's because they will likely be reused for the next paket etc. But its not as trivial as it might sound. There's more to routing than just holding the routing table in memory. That part is easy and cheap but processing every packet against it in minimum time is what's costly.

Thats probably the reason why on a NPE-300 processor you can not stick more than 512MB of RAM because it would not be fast enough to handle it anyway. So then you go and buy a NPE-G1 or now a NPE-G2 and you end up with a few thousand CHF bill.

Now multiply this with number of ISP's and BGP routers they have and you see the picture. Big ISP's will take care of the core routers, whatsoever as its their core business. But the multihomed customers at the other side of the planet now has to buy a new router just because you added one route more into the table. This is the global effect.

well i wonder really, whats so bad about dsl ? xdsl is not enough specific, ghdsl, sdsl, hdsl .. that are all leased line version of dsl. if that troubles even the SC modem will trouble. i brought my link up .. and it stayed up for years, except as in front of TIX-1 the cables where under water.

using some adsl/vdsl is not so a good idea, as the uplink may be to slow,and the stability of those setup would shure end up in flapping. if the backbone of SC fails CES and even A/V- DSL variants will fail .. So this redundancy is not the right way.

btw, about expensive router, i dont think on the client side is an fullfeed needed. an second hand router will lower the bill and doing the job as well.

i got asked once from a client to have bgp on his cable connection and an ADSL connection .. the only thing he stopped bothering me was to give him an estimated price for the equipment ;-)

using an 2 uplink firewall like the symantec or whatever will do the job better, and an email relay in front on the isp side will send to both ip's. with dyndns services it would even work if both connection are dynamic. positive story, this will lead to loadsharing, negative about that sometimes the way the connection goes is not under controll. Like p2p programms which are open more than one connection to the same destination. but set to failover only will work in any cases.

Roger

Lukas Beeler wrote:

...

Now, even expensive FB-DIMM memory by vendors like HP and IBM only costs around 360 CHF for 4 GB. And even small two way x86 boxes max out at around 32 - 48 GB. Even if Cisco and Juniper charge 10x as much, that'd still be only 3600 CHF.

I understand that routers use ASICs and probably faster memory than servers, but i can't really imagine it to be a problem to pop 4GB memory into a router that's connected directly to the internet.

Now, where am i mistaken?

There still a lot of hardware around which is at the memory expansion limit. And (talking about Cisco) the IOS images don't tend to get smaller... So finally you end up replacing the whole router or NM-engine for some kilo-$ instead of a relative cheap memory upgrade...

I agree with Fredy's concerns about link stability and flapping, especially for residential services. BGP on DSL can although be deployed as backup solution or, if you're close enough to the "BBCS owner", as main link. The major problem you face there is getting a skilled person when you're in trouble... The help-desk guy/girl you get at the phone does usually don't even know how to spell B-G-P and will ask you why you have 4 of them :-)

Daniele

-- This message has been scanned for viruses and dangerous content by MailGate, and is believed to be clean.

---

swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog

Andreas Fink a écrit :

Thats probably the reason why on a NPE-300 processor you can not stick more than 512MB of RAM because it would not be fast enough to handle it anyway. So then you go and buy a NPE-G1 or now a NPE-G2 and you end up with a few thousand CHF bill.

Now multiply this with number of ISP's and BGP routers they have and you see the picture. Big ISP's will take care of the core routers, whatsoever as its their core business. But the multihomed customers at the other side of the planet now has to buy a new router just because you added one route more into the table. This is the global effect.

Hi all,

Talking about a small local ISP we helped put together, we used some relatively low-end pc's running OpenBSD+OpenBGPd on flashdist (openbsd read-only on DiskOnModule flash disk). This was about 3 years ago. The machine has a Celeron 2.4Ghz and 512Mb of RAM, and today, getting full routes from two upstreams, the machine is using 181Mb of RAM and itself routing about 300mbit/s of traffic ...

Try this with a three years old Cisco for less than CHF 2000.--

This hardware has been running flawlessly since beginning ...

I wouldn't recommend this setup for ultra-large networks (well ... why not ...). I mean as core BGP routers, why not, but probably not as edge. If you are getting close to fulling your available memory on your Cisco, you should try "offloading" the BGP work to a software router ... you can get quad core pc's with 4Gb of RAM and pci-express gbit for less than CHF 2000 ... buy two of them if you are worried ... we have the real world example ... it WORKS !

It is even getting easier today, have a look at Vyatta : http://www.vyatta.com/ or http://www.imagestream.com/, or try it yourself using OpenBGPd (highly recommended), Quagga or Xorp.

BGP is the routing protocol of choice in the international world. If your route is on xDSL and flaps, it has an effect to the whole world who uses full routing tables, That's a gazillon of routers out there and in that scenario it doesnt matter if you use a full or partial routing table. this is your choice on the outgoing path but on the incoming path everyone out there has to learn your'e unreachable / reachable over a certain path.

On 06.03.2009, at 00:20, Zorg 421 wrote:

Hey you're all implying BGP = full routing table. BGP with just a single default route on a cheap link may be a good idea. A proven L3 link failure detection mechanism implemented on a wide range of boxes (standardized!)

---

swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog