Network Topology

[spidey07]

ooohhh, good thread. Let's keep this up.

OP, are you still around?

I know i kinda took over your thread, but seriosly...this could be a really good thread if we keep it civil and talk to the points and best practices presented.

 

[crashish]

Originally posted by: spidey07

Originally posted by: crashish

Originally posted by: spidey07

Originally posted by: crashish
Configure a "host" with the proper hardware and software to be a "network" and suddenly you no longer have a "host". That's pretty much how Cisco started off, so what's the problem with doing that now?

ASIC - it's all done in hardware now. At the speeds today a processor based system can't cut it.

Tell that to the ISPs who use such systems. I'm sure they (we) will be shocked to hear it.

that's cool. You're one of "those guys" and I'm one of "those guys"

the debate over who has the "best" routing solution between the network guys and the OS guys will go on forever I believe. There are great merits to both sides.

It is a similar to a live forum that I recall - the gigabit ethernet vs. ATM. Excellent points presented by both sides. Just as there are great points presented in the unix vs. network hardware debate.

From what I see it is split currently. I would be more than happy to hear another viewpoint...as we/I have tried to keep this thread on track.

So seriously, I would like to learn about the other side and I'm not waving an e-penis here. Could you maybe touch upon the current state of ISPs and the use of unix vs. hardware IOS/JunOS.

I know it sounds like a "bait" but in all honesty I would really love to hear your opinion on the industry. Mainly because it changes so quickly.

I value your opinion so do please respond. Like I said earlier, this thread could get real fun if we stay open and I'm sure others could learn.

-edit- what I mean is this category could use a healthy dose of serious large scale discussion.

signed and confirmed "network bigot"

I'm probably not one of "those guys"... at least, not the guy that you think I am. There are, as you say, merits to both sides, and I am neither an advocate of Unix for every problem nor (Insert Hardware Vendor Here) for every problem, but an advocate of a specific solution to a specific problem.

My point was that some shops are better served by software solutions and my response was only sparked by n0cmonkey's interest and your flat out derision of Unix/software solutions. Not to mention that you were simply wrong that anything less than a Cisco can't "cut it."

 

[crashish]

Originally posted by: spidey07

Originally posted by: crashish
Configure a "host" with the proper hardware and software to be a "network" and suddenly you no longer have a "host". That's pretty much how Cisco started off, so what's the problem with doing that now?

ASIC - it's all done in hardware now. At the speeds today a processor based system can't cut it.

But you're right. A router is nothing more than a super fast host that routes. But doing it in programmable hardware is best IMHO. Plus there is that whole design aspect - never let a host or end node do routing....leads to bad mojo.

I guess I'll reply to your edits now...
Design a system as a router instead of a host and there's no "bad mojo." Design a system as a host and then as an afterthought, throw in routing, sure, you'll have some issues.

 

[crashish]

Originally posted by: MrDudeMan

Originally posted by: blemoine
Spidey07: Do you really expect anyone to believe all that mumbo jumbo & nonsense?

Inlinefour. They network with magic network elves that move faster than the speed of light and personally handle every network request. Thats how they know if your doing something illegal or not. and that my friend is the truth.

please explain what was not to be believed. his answer was accurate and concise. if you didnt understand the vocabulary or concept, that doesnt give you the right to hate on him. he definitely knows what he is talking about.

inline, they connect together through a series of high speed routers all over the world. the topologoy, IIRC, is a star network, meaning every router is connected to every other router so if one fails an alternate route can be established. each ISP is part of a MAN (metropolitan area network) and also has its own LAN (subscribers). as far as the topology in a city for each ISP, i have no clue but i would imagine there is at least minimal redundancy.

basically there is no single entity we all connect to and call the internet. in classes it is usually denoted as the "cloud" from which connectivity is established.

Don't mean to nitpick here, but to (hopefully) clarify for the less knowledgeable. Your stated definition of a star is wrong, and I would say your recollection is wrong.

The Internet (big I) as a whole is an f'd up hybrid that aspires to be a true mesh. Break down into smaller segments and you will see clearer examples of specific topologies.

The meshing, though, is what yields the automagic alternate route finding.

 

[Boscoh]

Originally posted by: crashish

Originally posted by: MrDudeMan

Originally posted by: blemoine
Spidey07: Do you really expect anyone to believe all that mumbo jumbo & nonsense?

Inlinefour. They network with magic network elves that move faster than the speed of light and personally handle every network request. Thats how they know if your doing something illegal or not. and that my friend is the truth.

please explain what was not to be believed. his answer was accurate and concise. if you didnt understand the vocabulary or concept, that doesnt give you the right to hate on him. he definitely knows what he is talking about.

inline, they connect together through a series of high speed routers all over the world. the topologoy, IIRC, is a star network, meaning every router is connected to every other router so if one fails an alternate route can be established. each ISP is part of a MAN (metropolitan area network) and also has its own LAN (subscribers). as far as the topology in a city for each ISP, i have no clue but i would imagine there is at least minimal redundancy.

basically there is no single entity we all connect to and call the internet. in classes it is usually denoted as the "cloud" from which connectivity is established.

Don't mean to nitpick here, but to (hopefully) clarify for the less knowledgeable. Your stated definition of a star is wrong, and I would say your recollection is wrong.

The Internet (big I) as a whole is an f'd up hybrid that aspires to be a true mesh. Break down into smaller segments and you will see clearer examples of specific topologies.

The meshing, though, is what yields the automagic alternate route finding.

QFT. A star is nothing more than a hub-and-spoke, or wagon wheel, design. Where every end node connects back to the same central hub.

A network where everything has redundant paths to everything else is a fully meshed network.

I'll also chime in and say that I'm also very interested to hear about non Cisco/Juniper type systems being used for high-speed routing. Shoot, this'll help me a lot in the job I'll hopefully be getting soon.

 

[crashish]

Originally posted by: Boscoh

Originally posted by: crashish

Originally posted by: MrDudeMan

Originally posted by: blemoine
Spidey07: Do you really expect anyone to believe all that mumbo jumbo & nonsense?

Inlinefour. They network with magic network elves that move faster than the speed of light and personally handle every network request. Thats how they know if your doing something illegal or not. and that my friend is the truth.

please explain what was not to be believed. his answer was accurate and concise. if you didnt understand the vocabulary or concept, that doesnt give you the right to hate on him. he definitely knows what he is talking about.

inline, they connect together through a series of high speed routers all over the world. the topologoy, IIRC, is a star network, meaning every router is connected to every other router so if one fails an alternate route can be established. each ISP is part of a MAN (metropolitan area network) and also has its own LAN (subscribers). as far as the topology in a city for each ISP, i have no clue but i would imagine there is at least minimal redundancy.

basically there is no single entity we all connect to and call the internet. in classes it is usually denoted as the "cloud" from which connectivity is established.

Don't mean to nitpick here, but to (hopefully) clarify for the less knowledgeable. Your stated definition of a star is wrong, and I would say your recollection is wrong.

The Internet (big I) as a whole is an f'd up hybrid that aspires to be a true mesh. Break down into smaller segments and you will see clearer examples of specific topologies.

The meshing, though, is what yields the automagic alternate route finding.

QFT. A star is nothing more than a hub-and-spoke, or wagon wheel, design. Where every end node connects back to the same central hub.

A network where everything has redundant paths to everything else is a fully meshed network.

I'll also chime in and say that I'm also very interested to hear about non Cisco/Juniper type systems being used for high-speed routing. Shoot, this'll help me a lot in the job I'll hopefully be getting soon.

Unfortunately I don't have any good bookmarks, but google is obviously the way to go. One specific place to look, though, is the quagga homepage. Through the information and links there I picked up a lot of very helpful knowledge. The FreeBSD-ISP mailing list is a good resource too, but a little broader in scope.

 

[Boscoh]

Originally posted by: crashish


Unfortunately I don't have any good bookmarks, but google is obviously the way to go. One specific place to look, though, is the quagga homepage. Through the information and links there I picked up a lot of very helpful knowledge. The FreeBSD-ISP mailing list is a good resource too, but a little broader in scope.

Thanks, I'll check it out.

 

[n0cmonkey]

Originally posted by: spidey07

Originally posted by: n0cmonkey
Has anyone messed with OpenBSD's OpenBGPd? It looks like DE-CIX is looking to upgrade to it. Slides from a presentation on it.

hey unix bigot. Let the network take care of the network, let your hosts be hosts. Big rule in network design.



I get into this all the time with unix bigots so please do tell me why a host should be actively involved with routing????

Gee, because we do it in hardware/asic and you don't because you can't without a unix box the size of a fridge. that damn OS overhead and all.

-ps- i am fully aware that at the core of IOS it is a unix kernal running things.

So, that's a no? I didn't advocate anything, I asked if anyone had experience with it. That's all.

From reading the slides, emails, and articles it seems like OpenBGPD can hold its own against Cisco for BGP, but I don't have experience with either OpenBGPD or Cisco's BGP, or Juniper's BGP, or Quagga/Zebra, or anything else.

According to the slides, [De-CIX]http://unduli.bsws.de/papers/nanog36/mgp00041.html[/L] ("one of the biggest european exhange points, located in Frankfurt, Germany") is looking to upgrade to OpenBGPD.

Anyhow, thanks for the comments. Most of the things you post are enlightening.

 

[randal]

Mesa Networks, the US' 5th largest WISP, uses Unix-based routing exclusively. They use Soekris boxes running a custom Linux kernel on the edge, a slightly more powerful version in their distribution to handle dynamic routes, and their core is an IBM server running Linux + Quagga. They have something ridiculous like 6000 wireless clients and from what I know they move ~300mbps on a heavy day ... all without a hitch. It has been running that way since their inception.

I have also heard super duper good things about OpenBGPd. The high point of it to me is the ridiculous amount of RAM and CPU that you can through at BGP routing tables at little to no cost. I have no opinion regarding it's actual pps throughput or otherwise.

*Disclaimer: I am a total Cisco whore. But I do make sweet love to FreeBSD regularly

 

[Madwand1]

Request for clarification -- is 300 Mb/s supposed to be impressive at this time? Aren't ISP's thinking bigger?

Verizon FiOS for example provides (one way) 15 Mb/s links at consumer-affordable rates. It would take 20 concurrent users downloading the latest Paris Hilton video, finding that it's got a malware payload, and then restoring off GDrive, etc., to fully use that capacity.

http://today.reuters.co.uk/news/NewsArt..._RTRIDST_0_OUKIN-UK-GOOGLE-STORAGE.XML

Capitalization fault? 300 MB/s would give some more headroom, giving you around 200 concurrent GDrive restorations over FiOS 15 Mb/s or around 1000 concurrent 3 Mb/s customer sessions.

My point? I'm either missing something, probably a lot, or these figures point to a state-of-the-current that doesn't look ahead enough to some active future competition, if not just wishful thinking on my part.

 

[spidey07]

Major links are multiple 10 and 40 Gigabits per second (OC-192 and OC-768, that's the SONET stuff I was talking about, the transport layer)

juniper routers
http://www.juniper.net/products/tseries/

cisco routers
http://www.cisco.com/en/US/products/ps5763/index.html

I forget what the packet per second on those guys are, but it is pretty gnarly.

-edit- although I don't now how many are actually using the 40 gig links. it's pretty pricey.

yikes! Just a single cisco OC-768 card is nearly half a million dollars. the 4 port OC-192 is 750,000.

 

[randal]

Originally posted by: Madwand1
Request for clarification -- is 300 Mb/s supposed to be impressive at this time? Aren't ISP's thinking bigger?

Verizon FiOS for example provides (one way) 15 Mb/s links at consumer-affordable rates. It would take 20 concurrent users downloading the latest Paris Hilton video, finding that it's got a malware payload, and then restoring off GDrive, etc., to fully use that capacity.

http://today.reuters.co.uk/news/NewsArt..._RTRIDST_0_OUKIN-UK-GOOGLE-STORAGE.XML

Capitalization fault? 300 MB/s would give some more headroom, giving you around 200 concurrent GDrive restorations over FiOS 15 Mb/s or around 1000 concurrent 3 Mb/s customer sessions.

My point? I'm either missing something, probably a lot, or these figures point to a state-of-the-current that doesn't look ahead enough to some active future competition, if not just wishful thinking on my part.

Hahaha no, 300mbps is not impressive. I'm just saying that they do it on super cheap ultra low power Soekris boxes and run just linux. For them, deploying Cisco units to all of their tower sites was just too expensive to get the feature set they needed - firewalling, routing, VPN termination, lots of remote management and testing tools etc. all pushed down way out in their network. Plus the Soekris are all solid state with no moving parts, not even fans so their reliability is ridiculous. And software updates are 100% free.

So yea, Unix-based stuff has some super awesome benefits, but I don't know if they have the throughput that the real guns do. Take this anecdote for what is - a simple observation - as I'm not big into advocacy

 

[spidey07]

awww, c'mon....I want to talk networks....feed me.

More posts. Let's talk design, let's talk layers, let's talk functional blocks/layers, let's talk BGP and MPLS (my strenght and weakness at the same time, my head swims everysingle time I get deep into those) and the global Internet.

i know we have plenty of people who want to learn more and I would love to learn more.


speak up!

I say again - the Internet is nothing more than a collection of autonomous systems interconnected via sharing reachibility information, much like a phone system. In fact, exactly like a phone system.

 

[spidey07]

Originally posted by: randal

Originally posted by: Madwand1
Request for clarification -- is 300 Mb/s supposed to be impressive at this time? Aren't ISP's thinking bigger?

Verizon FiOS for example provides (one way) 15 Mb/s links at consumer-affordable rates. It would take 20 concurrent users downloading the latest Paris Hilton video, finding that it's got a malware payload, and then restoring off GDrive, etc., to fully use that capacity.

http://today.reuters.co.uk/news/NewsArt..._RTRIDST_0_OUKIN-UK-GOOGLE-STORAGE.XML

Capitalization fault? 300 MB/s would give some more headroom, giving you around 200 concurrent GDrive restorations over FiOS 15 Mb/s or around 1000 concurrent 3 Mb/s customer sessions.

My point? I'm either missing something, probably a lot, or these figures point to a state-of-the-current that doesn't look ahead enough to some active future competition, if not just wishful thinking on my part.

Hahaha no, 300mbps is not impressive. I'm just saying that they do it on super cheap ultra low power Soekris boxes and run just linux. For them, deploying Cisco units to all of their tower sites was just too expensive to get the feature set they needed - firewalling, routing, VPN termination, lots of remote management and testing tools etc. all pushed down way out in their network. Plus the Soekris are all solid state with no moving parts, not even fans so their reliability is ridiculous. And software updates are 100% free.

So yea, Unix-based stuff has some super awesome benefits, but I don't know if they have the throughput that the real guns do. Take this anecdote for what is - a simple observation - as I'm not big into advocacy

Well if they can push those kinds of features to the edge then that is awesome. Let the core/distribution layer be what it is and pushing those kinds of features out to the edge is nice. Especially if it is all solid state.

 

[Woodie]

alright Spidey...

Aren't there physical system limitations involved w/ using a host (regardless of OS) to do routing, as opposed to hardware designed for this? Specifically, I'm thinking that a 24 or 48 port managed switch has like a 10 or 20 Gbit/s backplane, while a PCI bus (used to route traffic from one nic to another) is limited to like 133Mb/s?

(Correct me! Please! Tell me what bus these other platforms are using)

 

[spidey07]

Sure there are bus limitations. But big iron unix boxes use a proprietary mid-plane/bus.

Same with the big cisco/juniper boxes.

They just design their own - in the hundreds of gigabits/sec range.

 

[randal]

Originally posted by: Woodie
alright Spidey...

Aren't there physical system limitations involved w/ using a host (regardless of OS) to do routing, as opposed to hardware designed for this? Specifically, I'm thinking that a 24 or 48 port managed switch has like a 10 or 20 Gbit/s backplane, while a PCI bus (used to route traffic from one nic to another) is limited to like 133Mb/s?

(Correct me! Please! Tell me what bus these other platforms are using)

From http://www.pcguide.com/ref/mbsys/buses/funcBandwidth-c.html (via google)...

PCI Bus is (barring newer stuff) guaranteed to be 32-bits wide at 33MHz yielding a bandwidth of ~1056mbps. 64-bit PCI (64bits x 66MHz) gets up to ~4224mbps. When you think about it, those are not very large numbers. Especially when it comes to pure switching and not routing - 4gbps doesn't even get to "joke" status on a switch.

The bang/buck ratio of a 1gbps router on a PC, though, is pretty crazy. The problem is that is generally only ethernet - get to any other interface type and the costs get high very quickly, just like with "normal" routing gear.

In a pure ethernet, routing-only environment, a properly configured *nix box is absolutely the price/performance winner, as long as the only two requirements are A) price and B) performance. In my opinion, such machines lose out badly when anything else comes into play (reliability, support, knowledge base, etc).

I happen to know this because I am a network admin for a largish LAN party - which has an approaching $0 budget - and we have usde a FreeBSD machine as our core router without issue for several years ... on the same AMD K6-2 processor, even!

 

[spidey07]

bump