Spidey's net project

[Mucman]

Originally posted by: spidey07
Mucman,

I was hoping we could keep a running thread of progress/decisions. I'm sure all the heavy hitters are salivating and were all friends here so we can argue all we want!

:beer:

I just wish I could keep up with all the stuff you are discussing so I could interject my opinion! I may have learned a lot about networking, and tooling around with the Cisco IOS, but I don't have a clue about the hardware differences between a PacketBlaster 2112 vs. the MassivePipe BlingSwitch... Been listening to too much Rush in Rio I guess .

<1337 c1u3n355>
Setup a 486 running OpenBSD and Zebra, and be sure to lay as much copper outside that you can! The minerals in the soil and occasional lightning blast will give you Terabit sp33dz0rz!!!! w3333333333
</1337 c1u3n355>

 

[Darthkim]

Sup Spidey,

Glad to see someone else buying the sup720. I am getting mine installed next week. From the sounds of it, i think we bought the same blades. Let me know how the install goes.

One hiccup that we did have is that with 2 fully loaded sup720, you need at least 4000W power supplies to have redundant power supplies. The 2000W psu will cut it, but they have to run together.

Good Luck!

 

[spidey07]

Originally posted by: Darthkim
Sup Spidey,

Glad to see someone else buying the sup720. I am getting mine installed next week. From the sounds of it, i think we bought the same blades. Let me know how the install goes.

One hiccup that we did have is that with 2 fully loaded sup720, you need at least 4000W power supplies to have redundant power supplies. The 2000W psu will cut it, but they have to run together.

Good Luck!

thanks for the tip! I didn't know a 6500 needed a 4000 watt supply for two sups.

 

[bgroff]

Originally posted by: spidey07

Originally posted by: Darthkim
Sup Spidey,

Glad to see someone else buying the sup720. I am getting mine installed next week. From the sounds of it, i think we bought the same blades. Let me know how the install goes.

One hiccup that we did have is that with 2 fully loaded sup720, you need at least 4000W power supplies to have redundant power supplies. The 2000W psu will cut it, but they have to run together.

Good Luck!

thanks for the tip! I didn't know a 6500 needed a 4000 watt supply for two sups.

Well, it depends. If you have a 6509 and you are using the 2500W supplies and are connecting the supplies to a 220V line, then you should be fine with dual sup720s. You just have to work your power budget for the line cards. Its quite a challenge to track down on Cisco's website, but you can find the power requirements for the sups and line cards here and you can find the amperage supplied by the various power supplies here. From the looks of it, the only way you'd need 4000W supplies on a 6509 is if you had 2 sup720s and 7 WS-X6748-GE-TX line cards...

 

[spidey07]

I've been told that the 4000 watters are really only for power over ethernet applications?

 

[saimike]

does it matter that the 6509 is gonna EOL soon? this is what i heard.

 

[bgroff]

Originally posted by: spidey07
I've been told that the 4000 watters are really only for power over ethernet applications?

Well, you really need it for a fully populated 6513 if you want redundant power supplies and have power hungry line cards.

 

[bgroff]

Originally posted by: saimike
does it matter that the 6509 is gonna EOL soon? this is what i heard.

Unless they come out with something bigger/better/faster I can't imagine this being the case. I don't see anything in the Cisco product line up that might conceivably replace the 6509...

 

[Darthkim]

bgruff,

we have 2 of the 6816 and 4 of 6748 cards. Now that you mention it, what caused us to go over to the 4000 PSU was the dcef cards for the 6748. I guess the additional power draw was more than the 2500's could handle (they can handle it, but you will essentially have 1 big psu (2 2500 bonded togeter). If one were to fail, some very funky things will happen.

What happened is that i spec'ed out this setup way back in august. Since the 6748 were just coming out and the dcef cards were not out yet, no one knew about the additional power draw. It wasn't until the dcef cards arrived late in january (delayed Multiple times), that we found out about the problem.

Thanks for bringing that up bgruff.

Also, the 6509 are far from being EOL. Also the 10 gig figured posted earlier are incorrect. The ones i remember reading a while back stated that the dual 10gig cards can handle everything at wire rate. Only the 4 10 gig cards could not (it can up to 98 percent ).

I also saw some testing done with the ixia equipment which demonstrated that the 6748 with dcef really could ouput 40gig to the backplane and thene some (well.. the ixia could do 48, but the cisco can't )

 

[nightowl]

Spidey, your project sounds a lot like the one I am working on right now. I am only get to implement the access layer rather than the core but it is still pretty neat to see. I my case we have 4 6509's with redundant sup720s connected in a ring. Connecting the ring is 10G ethernet and a 2G etherchannel . Going out to all of the buildings (this is a college campus) are 24 port LC gbic blades. Each building will be connected with a 2G etherchannel (most buildings will have a 4507 in the bdf). Definately cool stuff. Also, for whoever mentioned Juniper, we have 2 M10's taking care of the Internet connections Are you going to be doing any channeling of interfaces at all for redundancy or are you going to have routing taking care of that. If you are using channels there are some issues with using LACP for the channels? Also, as far as the 6509 power issue. We have the 2500W DC PS running in redundant mode and are using about 1700W with 3 24 port LC gbic blades, 2 sup720's, and a 4 port 10G card.

 

[cmetz]

nightowl, how do you like the M10? I've worked with 20s and 40s, but never actually got my hands on 5s or 10s. They've always been kind of red-headed stepchildren for Juniper, their sales people certainly don't seem to know how to sell them. It's a shame, because they look like they'd be great boxes for the 7200ish niche. (JunOS is soooo much better than IOS, IMO)

 

[Boscoh]

Seeing how I'm just a lowly CCNA working towards my CCSP and CCNP and should have both by the time I'm done with college (about 2-3 more years), I'm really enjoying this conversation. I dont have any experience in a big datacenter...a 15-server, 100-user, 4-site network is the biggest I've worked with. However, I do want to throw something out for the sake of curiosity as I've been thinking about it lately.

Spidey, you use Spanning-Tree on your network right? If you have multiple VLANs, couldnt you loadbalance the VLANs across Spanning-Tree primary and secondary links? Example....I've got Gig0/1 on my switches setup as the primary link, Gig0/2 is in Alternate-Blocking mode and I've got 4 VLAN's on the network. Couldn't I setup VLAN 1 and 3 to use Gig 0/1 as the primary and Gig 0/2 as the alternate, then setup VLAN 2 and 4 to use Gig0/2 as primary and Gig0/1 as alternate? That would keep both links forwarding, and would load balance your traffic, but still give you redundancy. I know that doesnt help you with your L3 issues in your datacenter...but I just wanted to throw that out there. It seems to me that it might give you more efficient use of some of your bandwidth. You might not even be using VLANs...I dunno. I've heard from a few people that Cisco is going to get rid of VLANs by 2006. Supposedly VLAN's arent even covered on the new multilayer switching exam that's in testing right now.

I agree that you should put pressure on your Cisco rep. Even in our relatively small business, I told our rep that we were evaluating Dell and 3Com switches when we were testing for a "big" network upgrade and that we already had a Dell, had tested it extensively and were very happy with it. That alone got them to bump up our discount. We were getting around 38% before, after I told him that, we ended up getting 4 C2950's, and 6 2940's for the price of 2 C2950's. You do the math...but they basically gave them away. Apparently Cisco REALLY REALLY has it in for Dell.

 

[spidey07]

Boscoh,

Yes. Unfortunately there are about 200 trees on the campus and they span every switch through trunks.

Bad mojo. Really bad. Poor design but that is how you did it 4.5 years ago when I put it in. Sure I did all the good stuf like managing my root bridges, etc but still there is no good reason to have trees span that many switches.

Hence why everything will be layer 3 and I'll let my OSPF experience take over. One of my design goals is to actually/almost eliminate the need for spanning-tree

 

[Boscoh]

So you're going to L3 everything from the core to the access layer? Interesting. OSPF would, I guess, be more managable in the long run...especially if you are a guru with it.

 

[spidey07]

Originally posted by: Boscoh
So you're going to L3 everything from the core to the access layer? Interesting. OSPF would, I guess, be more managable in the long run...especially if you are a guru with it.

core-distrubtion will be all layer3. more control, more managible, more redundant, faster convergence, easier to troubleshoot, etc. WAY too many good things.

Access layer to distribution will be layer2 but I'm going to try and eliminate redundant links relying on spanning-tree and use some form of channeling.

 

[nightowl]

Cmetz, I have not really had the chance to use them at all since I dealy almost exclusively with L2 & L3 switching. I don't do any of the BGP routing stuff (don't have any experience with that yet, only OSPF). They seem like good boxes though. They were put in to replace the 7500s that were in place to route the Internet links that we have. We also have 2 M5s at remote campuses too and from what I have seen they are basically a smaller M10 with less PICs. Other than that they are fairly similar, I believe.

Spidey, are you going to elminate all trunking inside the core then? We would like to be able to do that but everyone else will complain as we have subnets trunked all over the campus. There is too much resistence to change everything and security fears but that can be taken care of my simple firewalls or access lists. Also, what are you planning to use for channeling? We use channels extensively and that way you do not have to mess with spanning tree and you still have redundancy with load balancing.

 

[cmetz]

spanning tree bad. Replacing spanning tree with OSPFv2 is a very good idea. There are obvious network design requirements created, but, if you can do it...

 

[spidey07]

LOL!

spanning-tree bad.

Ever see your own OSPF hellos come back to you over and over again in a loop? Bad. Very bad. Network meltdown bad. Every switch/processor is pegged and four-pawed.

bad.

 

[SaigonK]

Spanning Tree......*shudder*

If you dont need a loop, why use it!

 

[Soybomb]

Out of curiosity, what sort of budget do you have for your project's hardware?

 

[spidey07]

.dp.

 

[spidey07]

350K for two switches plus some cabling hardware and labor.

 

[Darthkim]

We spent 300k on 1 fully loaded 6509, 2 4506 and 11 3750 (yes. L3 to the edge) and 24x7 4 hour smartnet

 

[spidey07]

Originally posted by: Darthkim
We spent 300k on 1 fully loaded 6509, 2 4506 and 11 3750 (yes. L3 to the edge) and 24x7 4 hour smartnet

He asked for budget. Not what it actually costs. Always shoot high for budget purposes.

 

[Buddha Bart]

Random CCNA question...

Do you guys actually mess with the various timers in OSPF, or its administrative distance? Or is everything generally left at the default?